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A  new,  highly  automated  method  for  measuring  plane  motion, 
pattern  mapping,  has  been  developed  for  rigid-body  motion  and  strain 
analysis.  Pattern  mapping  employed  image  processing  and  syntactic 
pattern  recognition  principles  to  recognize  a  known  pattern  before 
and  after  motion.  Using  the  Lagrangian  definition  of  motion,  points 
in  the  two  Images  were  mapped,  and  the  map  was  used  to  determine 
rigid-body  motion  and  strain.  Whole-image  analysis  was  thoroughly 
demonstrated. 

Known  patterns  were,  in  this  case,  rectangular  grids  of  photo- 
dots.  These  patterns  were  applied  to  the  specimen  by  contact  print- 
ing. An  icon  was  included  in  the  pattern,  thereby  accommodating 
rigid-body  rotations  up  to  one  revolution. 

Motion  functions  were  selected  by  the  user  and  easily  changed  to 
suit  the  particular  application.  Function  coefficients  were  deter- 
mined by  least  squares. 


XI 


Accuracy  depended  primarily  on  the  signal-to-noise  ratio  and  the 
gray  level  difference  between  "black"  and  "white."  An  increase  in 
these  variables  improved  accuracy. 

User  and  computer  time  requirements  were  quite  modest.  Images 
were  processed  with  little  or  no  user  interaction,  and  output  speci- 
fications were  automatically  read  from  a  disk  file.  Depending  on 
output  specifications,  computer  time  (VAX  11/750)  for  an  11X11 
pattern  in  a  384X374  image  was  25  to  38  CPU  seconds. 
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CHAPTER  I 
INTRODUCTION 


1 .1   Background 

The  primary  purpose  in  experimental  solid  mechanics  is  to  deter- 
mine stress  and  motion  (strain,  rigid-body  translation,  and  rotation) 
from  measurements  of  physical  quantities.  For  example,  stress  may  be 
computed  from  measurements  of  load  and  cross-sectional  area,  and 
strain  is  measured  by  changes  in  characteristic  lengths.  Many 
methods  (mechanical,  electrical,  optical,  etc.)  are  available  to 
analyze  stress  and  motion  [1-4]. 

Optical  methods  play  an  important  role  in  experimental  mechan- 
ics. They  have  a  number  of  advantages  which  include  [5,6] 

1 .  Noncontacting  measurements  do  not  interfere  with  material 
response. 

2.  Full-field  analysis  is  usually  possible. 

3.  Highly  accurate  results  are  usually  possible. 

4.  Response  times  are  as  fast  as  light,  thereby  accommodating 
dynamic  investigations. 

5.  Actual  structures  under  live  loads  can  frequently  be  ana- 
lyzed. 

6.  Effects  of  actual  boundary  conditions  can  be  studied  in 
detail. 


Although  optical  methods  offer  many  advantages,  they  do  have 
several  undesirable  features.  For  example,  some  methods  are  sensi- 
tive to  small  extraneous  vibrations  and  require  expensive,  delicate 
equipment.  Perhaps  the  most  undesirable  feature  is  the  fact  that 
most  of  these  methods  do  not  directly  yield  their  wealth  of  informa- 
tion about  stress  or  motion  in  a  readily  usable  form. 

The  extraction  of  this  information  is  time-consuming  and  re- 
quires special  skills.  An  expert  may  spend  hours  reducing  data  and, 
all  too  often,  the  amount  of  data  analyzed  is  limited  simply  because 
automated  analysis  is  not  available.  Furthermore,  results  can  be 
affected  by  the  investigator's  particular  techniques. 

Digital  computers  are  opening  doors  to  automation,  increased 
accuracy  of  existing  methods,  and  previously  intractable  or  unknown 
techniques.  Some  of  the  latest  innovations  employ  digital  image' 
processing  and  pattern  recognition  (IP/PR)  technologies  which  utilize 
digital  computers  and  video  imaging  equipment  for  direct  data  acqui- 
sition. 

The  fields  of  IP/PR  are  rapidly  expanding  and  are  well- 
established  in  many  areas.  In  experimental  mechanics,  their  poten- 
tial for  efficiently  gathering  vast  quantities  of  data  consistently 
and  accurately  is  being  quickly  developed.  These  technologies  are 
freeing  experimentalists  from  hours  of  painstaking  labor  and  provid- 
ing them  with  a  new  avenue  to  existing  numerical  methods  and  analyti- 
cal tools  [7-14]. 


I .2  Purpose  of  Present  Work 

The  IP/PR  technologies  have  their  price.  For  example,  IP/PR 
programs  that  use  fast  Fourier  transforms  or  correlation  (template 
matching)  are  frequently  considered  computationally  expensive  because 
they  entail  large  numbers  of  (relatively  slow)  floating-point  calcu- 
lations. Interactive  programs  provide  great  flexibility,  but  they 
need  the  user's  constant  attention.  Iterative  and  backtracking 
(systematically  retracing  previous  steps)  methods  can  require  sub- 
stantial amounts  of  computer  memory  for  intermediate  data  storage 
[15-18]. 

Significantly  reducing  these  time  and  memory  requirements  is 
certainly  a  desirable  goal.  Perhaps  this  is  possible  by  rethinking 
the  problem  and  developing  a  highly  automated  method  which  capital- 
izes on  the  strengths  of  the  digital  computer.  Thus,  the  purpose  of 
this  work  is  to  develop  such  a  method,  pattern  mapping,  to  obtain 
data  via  digital  imaging  equipment  and  subsequently  determine  plane 
motion. 

1 .3  Scope  of  Present  Work 

Although  it  was  not  apparent  in  the  beginning,  the  concepts 
developed  in  this  investigation  are  readily  applied  to  the  photodot 
(grid)  method  [19].  The  scope  includes  this  photodot  adaptation. 

Actually,  this  investigation  encompasses  a  variety  of  scientific 
disciplines.  Perhaps  the  best  approach  is  to  consider  the  attributes 
of  each  and  work  toward  a  goal  that  is  obtainable  in  a  reasonable 
amount  of  time.  Specific  objectives  included  in  the  scope  are  to 


1  .  guarantee  unique  mapping 

2.  provide  subpixel  registration  of  0.1  pixel 

3.  provide  a  wide  range  of  motion  measurement 

4.  provide  whole-image  analysis 

5.  reduce  computer  and  user  time  as  much  as  possible 

6.  provide  highly  automated  analysis, 

(Pixels  are  discrete  elements  in  a  digital  image,  and  whole-image 
implies  results  are  computed  for  every  photodot.)  Special  emphasis 
is  on  accuracy  and  increased  computation  efficiency. 

Several  laboratory  experiments  using  a  CR-39  (plastic)  prismatic 
beam  were  included  in  the  study.  The  experiments  were  translation, 
rotation,  general  rigid-body  motion,  four-point  bending,  and 
cantilever  beam.  All  experiments  were  static  cases  of  motion  in  a 
plane  normal  to  the  optical  axis.  A  rectangular  (11X11)  pattern  of 
photodots  was  contact  printed  on  the  lateral  side  of  the  beam. 

For  convenience,  motion  is  defined  only  in  terms  of  the 
Lagrangian  formulation.  Evaluation  of  motion  functions  are  limited 
to  a  linear  and  a  second  order  approximation.  The  least  squares 
method  of  curve  fitting  is  used  exclusively,  and  normal  equations  are 
solved  by  matrix  inversion  using  Gauss-Jordan  elimination. 

Synthetic  images  are  used  to  debug  analysis  programs  and  inves- 
tigate the  influence  of  various  factors.  Motion  in  the  images  in- 
clude affine  transformations  and  rigid-body  rotation  about  the  image 
center.  Interpolation  of  between-pixel  intensities  is  bilinear,  and 
system-induced  distortions  are  assumed  eliminated  during  preprocess- 
ing.   Noise  has  a  zero  mean  and  selectable  Gaussian  or  uniform 


distributions.   Image  size  is  256X256  pixels  which  are  unsigned,  16 
bits  or  less.  The  camera  position  is  assumed  fixed. 

Segmentation  and  feature  extraction  (dividing  the  image  into 
subregions  and  obtaining  useful  information)  include  neither  prefil- 
tering  nor  histogramming  (gray  levels)  and  are  based  on  the  random 
selection  of  a  few  pixels.  Processing  is  automatic  and  direct 
because  known  patterns  in  the  image  are  always  completely  visible  and 
recognizable.  Feature  space  includes  gray  levels,  areas,  centroids, 
and  second  moments.  The  pattern  language  consists  of  only  one  sen- 
tence, and  it  is  context-sensitive. 

1.4  Survey  of  Previous  Work 
1.4.1   Solid  Mechanics 

Early  applications  of  image  processing  in  experimental  mechanics 
involved  the  analyses  of  fringes.  The  fringes  were  from  photoelasti- 
city,  moire,  or  holographic  interferometry.  Burger  [7J  and  Chen  [8] 
describe  several  of  these  applications  in  detail. 

Another  image  processing  technique  applied  to  mechanics  is  cor- 
relation. Researchers  [9,20-24]  determine  plane  displacements  (u-, 
i=1,2)  and  their  gradients  (du^/dX-,  i,j=1,2)  by  comparing  digitally 
recorded  images  of  the  same  random  pattern.  For  example,  a  pattern 
of  black  speckles  (e.g.,  minute  specks  of  black  paint)  on  a  white 
field  is  recorded  as  integer  gray  levels  at  discrete  points 
(pixels).  Two  images  are  used  in  the  analysis:  one  before  and  one 
after  motion.  By  comparing  (spatially  registering)  small  subimages 
from  each,   displacements  and  their  gradients  are  determined  at 


selected  points  in  the  (original)  pattern.  Sigler  and  Haworth  [25] 
combine  holography  with  correlation  to  determine  motion.  Thus,  the 
correlation  method  has  the  advantages  of  using  coherent  or 
conventional  light  sources  and  easily  applied  patterns. 

The  grid  method  is  one  of  the  oldest  and  simplest  methods  used 
in  the  analysis  of  strain.  One  simply  applies  reference  marks 
(lines,  spots,  scratches,  etc.)  to  the  surface  of  a  specimen  and 
records  the  original  distances  between  marks.  Next,  the  specimen  is 
deformed  and  the  distances  are  again  recorded.  Then,  changes  in  the 
distances  are  determined  and  strain  is  computed  [1,2,26-32]. 

Although  simple,  the  grid  method  is  tedious  and  traditionally 
limited  to  larger  strains.  Typical  strain  measurements  are  greater 
than  0.001  inches-per-inch;  the  results  depend  on  the  care  exercised 
and  the  instrumentation  [19,26,28].  Even  when  optical  comparators 
are  used  for  the  smaller  strains,  many  man-hours  are  frequently 
needed  to  complete  the  analysis  of  large  grids. 

Examining  a  specimen  through  a  comparator  and  determining  the 
position  of  a  grid  mark  is  also  a  subjective  process.  Some  investi- 
gators define  a  mark's  position  by  its  left  edge,  some  by  its  right 
edge,  and  some  by  a  variety  of  definitions.  All  these  definitions 
are  subject  to  error  caused  by  rotation,  the  nonlinear  characteris- 
tics of  the  recording  medium  (if  photographically  recorded),  grid 
application  techniques,  lighting,  operator  fatigue,  etc.  [19]. 

Parks  [32]  suggested  computer  technology  might  improve  the 
accuracy  of   the  method  and  would   certainly  reduce  the  labor 


involved.   Previous  efforts  to  fully  computerize  the  grid  method  have 
not  been  reported  in  the  literature. 

Sevenhuijesen  [33,3^1  reported  some  feasibility  studies  with  a 
Reticon  photo-diode  array  camera.  Using  an  eight-bit  digitizer  and 
digitizing  the  signal  from  each  diode,  he  obtained  a  resolution  of  a 
few  hundred  micro  inches-per-inch.  Images  were  recorded  photographi- 
cally without  the  aid  of  an  image  processing  system. 
1.4.2  Image  Processing/Pattern  Recognition 

Several  researchers  active  in  IP/PR  have  been  interested  in  two- 
and  three-dimensional  motion  analysis.  Most  efforts  have  concentrat- 
ed on  the  motion  of  rigid  bodies,  clouds,  the  human  body,  optical 
flow,  etc.  Motion  parameters  were  obtained  by  a  variety  of  iterative 
processes  [35-38]. 

Huang  and  Tsai  [39-43]  proposed  a  direct  method  of  mapping  the 
three-dimensional  motion  of  a  plane  using  two,  time-sequential 
images.  Mapping  parameters  were  determined  uniquely  by  solving  a  set 
of  linear  equations.  Unfortunately,  motion  parallel  to  the  optical 
axis  could  not  be  determined,  and  "small"  approximations  were  made 
for  strain  and  rotation.  Mathematical  proofs  guaranteeing  unique 
mapping  between  images  were  included. 


CHAPTER  II 
THEORETICAL  FOUNDATIONS 


II.  1   Solid  Mechanics 


The  description  of  motion  used  here  is  limited  to  the  Lagrangian 
description  and  is  consistent  with  the  classical  field  theories  of 
modern  continuum  mechanics  [44-48].  The  Lagrangian  description  is 
sufficient  to  determine  the  directions  of  the  principal  axes  of 
strain  and  the  magnitudes  of  the  principal  stretches.  It  is,  there- 
fore, a  fully  general  measure  of  strain  and  is  capable  of  describing 
small  or  large  strains.  For  convenience,  all  reference  frames  are 
Cartesian  in  three-dimensional  Euclidean  space. 

In  Figure  1 ,  body  6  consists  of  a  set  of  particles  which  assume 
a  continuous  progression  of  configurations  in  time.  The  undeformed 
configuration  is  chosen  as  the  reference  state  where  particle  P 
occupies  the  "material"  position  X  at  time  t=tQ.  Subsequent  motion 
carries  particle  P  to  the  "spatial"  position  x  in  the  deformed  or 
"after  motion"  state  at  time  t.  Although  each  state  may  be  defined 
in  terms  of  its  own  coordinate  system,  material  and  spatial  coordi- 
nates are  usually  measured  with  respect  to  the  same  coordinate 
axis.  The  motion  of  a  point  is  described  by  the  relation  of  the 
coordinates  between  the  two  states.  Mathematically,  motion  is  ex- 
pressed by  a  one-to-one  mapping  x  symbolized  as 


X  =  x(X,t)   or   x^  =  X. (X^ .X^.X  ,t) 


(1) 


Figure  1 .  The  motion  of  body  g  through  space. 


The  deformation  gradient  of  the  motion  x  is  defined  as  the 
derivative  or  gradient  of  x  at  X  and  is  denoted  by 


-  "  3X 


or   F.  . 


3x. 

1 

3X. 


(2) 


where  F  is  the  fundamental  quantity  for  the  analysis  of  local  proper- 
ties of  deformation,  and  the  physical  components  are,  of  course, 
diraensionless  as  are  all  measures  of  relative  configuration. 
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Alternatively,  motion  written  in  terms  of  the  displacement 
vector  u  is 


X  =  X  +  u(X,t)   or   X.  =  X.  +  u.CX^.X^.X  t) 


(3) 


And  the  derivative  of  u,  the  displacement  gradient,  is 


3u 


3u. 

1 


^  =  3X   °^   ^ij  =  3X. 


(4) 


If  two  particles  in  the  reference  configuration  are  an  infini- 
tesimal distance  dX  apart,  then  these  two  particles  are  an  infinites- 
imal distance  dx  apart  in  the  deformed  configuration  (see  Figure 
2).     Tensor  F  maps   the   neighborhood   of   particle   P   in   the 


Figure  2.  The  motion  of  the  neighborhood  of  particle  P, 
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reference  configuration  to  its  neighborhood  in  the  deformed  configur- 
ation. Thus,  the  linear  transformation  is 


dx  =  F-dX   or   dx.  =  F. .dX.  (5) 

-     ~  1    ij  J 


Unique  mapping  is  guaranteed  as  long  as  the  Jacobian  J  =  |f|  is 
greater  than  zero  and  finite. 

Since  vector  dX  has  length  dS  and  vector  dx  has  length  ds, 
Lagrangian  strain  E  may  be  defined  in  terras  of  these  lengths  squared 
as 


(ds)^  -  (dS)^  =  2dX-E-dX  (6) 


The  Green  deformation  tensor  C,  referred  to  the  undeformed  con- 

figuration,  gives  the  new  squared  length  (ds)   of  the  element  into 

which  the  given  element  dX  is  deformed.   Symbolically,  the  relation- 
ship is 


(ds)^  =  dX-C'dX  (7) 


where 


C  =  f'^-F  (8) 


Conversely,  the  Cauchy  deformation  tensor  B   gives  the  initial 

2 
squared    length  (dS)  of    an    element  dx  in    the    deformed 
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configuration.  This  relationship  is  written  as 


(dS)^  =  dx-B  ^  -dx  (9) 


where 


-1     -1  T  -1 
B   =  (F   )  -F  (10) 


Lagrangian  strain  written  in  terms  of  the  Green  deformation 
tensor  is 


E  =  ^[C-l2  (11) 


In  terms  of  the  displacement  gradient,  E  is 


3u    3u  8u  9u 

p    _  J.r i.  +  J.  +  H  i.1 

ij    2^3X.    3X.  3X.  9X.-' 

J      1  1    J 


Strain,  computed  by  a  Taylor  series  [1],  is 


(12) 


^11  =  A/^"2g3^^%)  "^ax^  "^3x)  -' 


9u   9v   3u  3u   3v  ^   9w  9w 
„     „     1     .    ^dY   *   dX  ^   9X  9Y  ""  9X  9Y  ""  9X  9Y, 
^12  =  ^21  =  2  ^^^^^"^ (UE^^)(UE^^) ^ 
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For  many  engineering  applications,  the  so-called  "small  dis- 
placement" theory  is  adequate  and  much  easier  to  implement.  By  this 
theory,  strain  is 


^11   3X 


U3u   3V-, 

E  =  — r —  +  — 1 

12   2^9Y   3X-' 

(14) 


e   =  £ 
21     12 


=  IX 
^22   9X 


If  the  Jacobian  J  is  greater  than  zero  and  finite,  the  deforma- 
tion gradient  written  in  terms  of  a  rotation  tensor  R  and  the  right 
stretch  tensor  U,  a  polar  decomposition,  is 

F  =  R'U  (15) 

Tensors  R  and  U  are  unique  and  represent  rigid-body  rotation  and 
stretch,  respectively.  Rotation  tensor  R  is  orthogonal  and  produces 
a  rigid-body  rotation  between  the  principal  axes  of  C  at  X  and  the 
principal  axes  of  B  at  x.  The  necessary  and  sufficient  condition 
for  no  local  rigid-body  rotation  (sometimes  called  "pure  strain") 
during  motion  is  R=1  .  Symmetric,  positive-definite  U  produces  the 
changes  in  vector  lengths  during  motion  and  produces  rotation,  in 
addition   to  R,  of   all   vectors   except   those   in   the   principal 
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directions  of  U.   Note  that  U=l   is  necessary  and  sufficient  to  define 
locally  "pure"  rigid-body  motion  (neglecting  the  trivial  case  when  no 
motion  occurs),  and  translation  does  not  change  a  vector  with  respect 
to  the  common  reference  axis.  Tensors  U  and  R,  written  in  terms  of 
F  and  C,  are 


y  =  c^^^  (16) 


R  =  l'^~^  (17) 


As  a  final  note,  the  deformation  at  any  point  may  result  from 
translation,  a  rigid-body  rotation  of  the  principal  axes  of  strain, 
and  stretches  along  these  axes.  These  motions  may  occur  in  any 
order,  but  their  tensorial  measures  may  not.  Mathematically, 
Lagrangian  motion  is  assumed  as  a  successive  application  of 

1 .  stretch  by  U 

2.  rigid-body  rotation  by  R 

3.  translation  to  x. 

II -2  Image  Processing  and  Pattern  Recognition 
II. 2.1   Digital  Images 

A  typical  image  processing  system  is  shown  in  Figure  3.  Conver- 
sion of  the  intensity  distribution  of  a  camera  image  to  a  form  suit- 
able for  computers,  digitizing,  is  accomplished  by  the  digitizer. 
After    digitization,    the    image    is    transferred    to    the 
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computer    for   further   processing.      How  an   image   is  processed  depends, 
of  course,    on  the  hardware  configuration  [15,18,35,49-51]. 


Object 


CCD  Camera 


[ 
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Microcomputer 
«lch  Internal 
Digitizer 


Digitizer 
Monitor 


Figure  3.   A  typical  PC-based  image  processing  system 


A  digital  image  is  an  approximation  of  a  real,  continuous 
image.  For  example,  the  real  image  of  the  spot  in  Figure  Ha  is  re- 
presented in  digital  form  as  the  matrix  of  integer  numbers  in  Figure 
4b.  Each  matrix  element  (or  pixel  which  is  dimensionless)  corre- 
sponds to  a  discrete  point  in  the  real  image,  and  the  value  of  the 
element  (gray  level)  is  the  approximation  of  the  average  local  illum- 
ination intensity.    Notice  that  black  has  lower  gray  levels  than 
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white  and  that  the  edge  region  (boundary)  between  the  two  has  inter- 
mediate values.  The  variation  of  gray  levels  in  regions  of  constant 
illumination  is  the  effect  of  noise.  Gray  levels  range  from  zero  to 
some  maximum  which  depends  on  the  hardware  (e.g.,  camera,  digitizer). 
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Figure  4.  A  spot  and  its  digital  image.  The  values,  I(x,y),  in  the 
digital  image  correspond  to  illumination  intensity  in  the 
real  image. 


Noise  corrupts  the  image  gray  levels,  and  the  sources  may  be 
classified  into  two  groups:  optical  and  electronic.  Two  examples  of 
optical  noise  are  particulate  matter  in  the  air  and  laser  speckle. 
Electronic  noise  comes  from  the  discretization  process,   thermal 
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sources  in  the  hardware,  interference  from  extraneous  electromagnetic 
sources,  etc.  Noise  is  usually  assumed  statistically  random  and 
normally  distributed  with  a  zero  mean,  u,  and  a  standard  devia- 
tion, 0^.  Unfortunately,  no  known  method  completely  eliminates  the 
effects  of  noise. 

Each  array  element  has  neighbors  which  are  commonly  called  the 
eight-neighbors  [52].  In  Figure  5,  the  center  element,  pixel  (x,y), 
is  the  reference  element  or  element  of  interest.  The  element 
directly  above  it  is  arbitrarily  labeled  neighbor  0,  and  the  other 
elements  are  named  in  a  counterclockwise  direction.  Although  ele- 
ments  along   the   image's   outer   array   edge  do  not  have  all 
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0 
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(x.y) 
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5 

Figure  5.  A  pixel  and  its  eight-neighbors. 


8  neighbors,  the  neighbors  they  do  have  are  also  labeled  according  to 
Figure  5. 

Spatial  and  illumination  sampling  must  be  sufficient  to  repre- 
sent adequately  the  real  image.  Too  low  a  spatial  sampling  frequency 
averages  intensity  over  too  large  an  area  (see  Figure  2.7  in  refer- 
ence 15),  while  too  high  a  sampling  frequency  results  in  the  loss  of 
information  outside  the  field  of  view.  In  general,  increasing  inten- 
sity corresponds  to  increased  gray  levels,  but  intensities  beyond  the 
capability  of  the  equipment  are  limited  to  the  maximum  gray  level. 
An  increase  in  the  range  of  gray  levels  Increases  image  detail  (see 
Figure  2.8  in  reference  15). 

11. 2. 2  Preprocessing 

After  digitization,  the  digital  image  is  corrected  for  geometric 
and  radiometric  distortion.  Distortion,  introduced  by  the  processing 
system,  is  removed  by  using  the  appropriate  calibration  procedure 
[^^9,53].  This  type  of  correction  is  frequently  called  preprocessing 
and  images  in  this  study  are  assumed  corrected  unless  otherwise 
stated. 

11. 2. 3  Segmentation 

The  next  step  in  image  processing,  segmentation,  consists  of 
identifying  regions  and  boundaries  of  interest  in  the  image 
[18,54].  Of  course,  the  difficulty  in  segmenting  an  image  depends  on 
its  complexity.  Since  the  images  developed  in  this  study  are  rela- 
tively simple,  they  are  easily  segmented. 

Histogramming.  Segmentation  usually  begins  with  histogram- 
ming.  When  the  frequency  distribution  of  the  gray  levels  in  an  image 
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i3  computed  for  the  entire  image,  contrast  and  the  presence  of  multi- 
ple modes  are  readily  observed.  Figure  6,  the  bi-modal  histogram  of 
Figure  4,  has  an  average  black  of  50  and  an  average  white  of  150. 
Note,  the  histogram  yields  no  direct  information  about  the  location 
of  the  gray  levels  in  the  image  [55]. 
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Figure  6.  The  bi-modal  histogram  of  Figure  4, 


Usually,  computing  the  histogram  for  the  entire  image  is  not 
considered  computationally  expensive.  This  is  not  true  in  the 
present  example  because  the  percent  of  the  total  computer  time 
devoted  to  histogramming  the  entire  image  is  significant.  Since  the 
images  used  here   are  designed  to  have  histograms  similar  to  Figure  6, 
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the  needed  information  is  easily  estimated  without  actually  computing 
a  histogram. 

Thresholding.  Thresholding  follows  histogramming  and  includes 
the  task  of  determining  the  gray  level  which,  in  this  case,  reason- 
ably separates  regions  of  black  from  white.  This  is  frequently 
accomplished  interactively  or  by  use  of  a  priori  information,  and  the 
choice  of  the  optimum  threshold  is  a  subject  of  considerable  interest 
[56-62]. 

The  alternative  approach  in  this  study  requires  neither  inter- 
active thresholding  nor  an  optimum  threshold.  Instead,  a  good  esti- 
mate of  gray  levels  that  are  "definitely  black"  and  an  estimate  of 
the  average  white  level  are  used.  The  estimates  are  derived  from 
histogram  statistics  and  based  on  the  following  assumption:  the 
histogram  is  strongly  bi-modal.  In  Figure  6,  gray  level  53  is 
assumed  black  and  the  average  white  gray  level  is  about  151  .  The 
task  of  estimating  these  levels  is  a  four-step  process. 

First,  pixels  are  randomly  selected  and  their  average  gray  level 
I.  is  computed  along  with  the  standard  deviation  o..  A  sample  size 
of  30  or  more  is  statistically  sufficient  to  estimate  the  mean  and 
standard  deviation  of  the  entire  histogram  [63,64].  Obviously, 
darker  pixels  are  to  the  left  of  the  average  and  lighter  pixels  are 
to  the  right  (see  Figure  7). 

In  the  second  step,  the  standard  deviation  is  subtracted  from 
the  average,  resulting  in  Iq.  Algebraically,  this  is 
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Figure  7.  Black  and  white  regions  in  a  bi-modal  histogram  (1^=132, 
13=98,  1^=53,  l2=151 ,  1^=102). 


Another  set  of  30  or  more  pixels  whose  values  are  below  Ig  are  ran- 
domly selected  and  used  to  compute  the  average  I^ .  Thus,  the  average 
black  level  is  approximated  by  I^ . 

Chebychev's  theorem  [53]  states  that,  if  a  probability  distribu- 
tion has  a  mean  vi  and  a  standard  deviation  0,  the  probability  of 
obtaining  a  value  x  which  deviates  from  the  mean  by  at  least  K  stand- 
ard deviations  is  at  most  —  .   Mathematically,  this  is  expressed  as 

K 


P(|x-ii|  >  Ka)  <  ^ 
K 


(19) 
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Since  only  one  standard  deviation  is  subtracted  from  the  average, 
finding  pixels  darker  than  Ig  is  highly  probable.  A  SKew  of  the 
histogram  to  the  left  further  improves  the  odds  of  finding  pixels 
less  than  Ig. 

The  third  step  is  the  initial  estimation  of  the  average  white 
gray  level,  Ip.  A  sufficient  random  sample  whose  values  are  greater 
than  I.  is  used  to  compute  the  initial  estimate  of  I2. 

Finally,  the  threshold  value  I^  is  computed  using  the  estimates 
for  the  black  and  white  levels.  Symbolically, 


I^  =  ^T-^  (20) 


Images  designed  for  this  study  are  assumed  to  have  two 
regions:  black  and  white.  In  a  sense,  they  are  binary  (region) 
images.  Thus,  segmentation  is  easily  accomplished  by  locating  and 
isolating  the  black  regions  of  interest  from  the  white  background. 

Raster  scanning  and  border  following.  Beginning  at  the  origin, 
the  upper  left  corner,  the  image  is  raster  scanned  (i.e.,  along  a  row 
or  column)  until  a  black  pixel  is  found.  If  at  least  one  of  the 
neighbors  is  black,  a  possible  black  region  is  assumed — spurious 
noise  is  assumed  if  the  pixel  has  no  black  neighbors.  If  a  black 
neighbor  is  found,  the  border  between  black  and  white  is  followed 
around  the  black  region  thereby  isolating  it  from  the  neighboring 
white. 
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Rosenfeld  [18]  has  developed  a  general -purpose  border-following 
algorithm.  First,  gray  levels  are  reduced  by  thresholding  to  O's  for 
black  or  I's  for  white.  (This  is  quickly  done  if  one  has  the 
appropriate  hardware.)  By  comparing  neighbors  and  locating  the  next 
edge  pixel,  the  algorithm  follows  the  border  counterclockwise  for  an 
outer  border  and  clockwise  for  an  inner  one.  Since  borders  may 
overlap,  provisions  are  made  for  backtracking. 

In  this  study,  the  generality  of  Rosenfeld' s  algorithm  is  un- 
necessary, and  the  algorithm  is  modified  for  faster  execution.  The 
modified  algorithm  (see  appendix)  is  designed  to  follow  outer  edges 
(no  inner  edges  exist)  in  the  clockwise  direction,  and  reducing  the 
image  is  eliminated  by  combining  thresholding  with  border  following. 

The  only  information  saved  during  border  following  is  the  y 
coordinate  of  the  spot's  left-most  pixel,  L;  the  y  coordinate  of  the 
right-most  pixel,  R;  the  x  coordinate  of  the  highest  pixel,  T;  and 
the  X  coordinate  of  the  lowest  pixel,  B.  Since  the  black  region 
extends  a  little  beyond  the  approximate  border,  2  pixels  are  sub- 
tracted from  L  and  T,  while  2  pixels  are  added  to  the  R  and  B  (see 
Figure  8).  Thus,  the  local  region  of  interest  (black  and  inter- 
mediate gray)  is  "framed"  for  further  processing. 

Raster  scanning  resumes  at  the  bottom  of  the  frame  and  continues 
until  another  black  region  is  found  and  it  is  framed.  Scanning  and 
framing  continue  until  the  entire  image  is  covered.  Thus,  the  image 
is  completely  segmented  and  the  information  needed  for  feature  ex- 
traction is  available. 
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Figure  8.  Frame  of  spot  in  Figure  4 


II. 2. 4  Image  Features 

Images  contain  various  types  of  information  known  as  features, 
and,  when  properly  selected,  they  adequately  represent  the  images. 
Here,  it  is  assumed  that  natural  features  are  intrinsic  to  the  real 
scene  and  artifacts  are  added  as  desired  to  aid  subsequent  proces- 
sing. Features,  both  natural  and  artifact,  are  generally  grouped 
into  three  classes:   physical,  structural,  and  mathematical  [65]. 
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Physical  and  structural  features  are,  in  this  case,  the  arti- 
facts shown  in  Figure  9  plus  the  pattern  they  form  in  the  image.  For 
simplicity,  the  (single)  icon  is  considered  a  "special"  spot,  and  all 
other  spots  are  assumed  uniform  in  size. 


Spot 


Icon 


Figure  9.   Image  artifacts.  The  icon  is  a  "special"  spot, 


Mathematical  features  are  frame  areas,  spot  centroids,  and  those 
concerning  icon  identification  and  rotation.  These  features  allow 
extensive  integer  programming  which  speeds  the  majority  of  the  cal- 
culations and  improves  accuracy. 
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II. 2. 5  Feature  Extraction 

Here,  feature  extraction  consists  of  isolating  the  artifacts  and 
computing  their  desired  mathematical  features.  Since  the  artifacts 
are  extracted  (by  framing)  during  segmentation,  all  that  remains  is 
the  computation. 

Frame  area.  Frame  (integer)  area  is  easily  computed  using  the 
information  obtained  during  the  segmentation  process.  The  computa- 
tion is 

A  =  (R-L)»(B-T)  (21) 

Since  the  icon's  frame  area  is  much  larger  than  the  other  spots', 
classifying  each  artifact  as  "spot"  or  "icon"  is  a  simple,  binary 
task. 

Spot  centroids.  The  centroids  of  all  spots  are  used  as  defini- 
tions of  spot  position.  If  the  gray  levels  in  a  spot's  frame  are 
assumed  to  be  "weights,"  the  x  and  y  coordinates  of  the  centroid  are 
calculated  [18]  by 


B   R 

E   X  xl(x,y) 
X  -^izTLy^L 


B   R 
I   I   I(x,y) 
x=T  y=L 


(22) 
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Henceforth,   these  summation  limits  are  implied  unless  otherwise 
noted. 

How  do  gray  levels,  noise,  and  spot  size  affect  spatial  resolu- 
tion of  the  centroid?  Here,  it  is  evident  (see  Figure  8)  that  merely- 
summing  with  gray  levels  I(x,y)  does  not  yield  the  proper  result 
because  the  white  pixels  shift  the  spot's  computed  centroid  away  from 
the  correct  value.  A  transformation  of  all  gray  levels  in  the  frame 
by  the  equation 


I'(x,y)  =  I  -I(x,y),  (23) 

w 


where  I^  is  the  local  average  white  level  and  I'(x,y)  is  the  noisy 
gray  level  difference  (between  black  and  white),  gives  the  approxi- 
mate centroid  coordinates.  And  since  pixels  on  the  frame  edge  are 
white,  their  average  gives  a  good,  local  estimate  for  I^^.  The  sub- 
stitution of  I'(x,y)  into  equations  (22)  leads  to 


Z  I  X  I'(x,y) 

y  ^  x_y 

E  I  I'(x,y) 
X  y 

(24) 


I   Z   y   I'(x,y) 

Y  =  y  X 

I   E  I'(x,y) 
y  X 


28 


To  account  for  noise,  assume  each  gray  level  in  the  image  con- 
sists of  the  actual  (i.e.,  uncorrupted  by  noise)  value,  r(x,y),  plus 
zero-mean  Gaussian  noise,  ri(x,y).   Thus, 


I'(x,y)  =  I  -  r(x,y)  +  nCx.y) 
w 


I'(x,y)  =  H(x,y)  +  n(x,y)  (25) 

where  H(x,y)  =  I  -  r(x,y)  is   the   actual   gray   level   difference, 
w 

Substituting  equation  (25)  into  equations  (24)  leads  to 

IIxH  +  ZZxn 
-  _  xy     xy 


lEH  +  IZt\ 
xy    xy 


(26) 


ZZyH  +  ZZyn 
-  ^  yx     yx 
ZZH  +  ZZn 
yx    yx 

If  the  expected  value  [63]  for  the  noise  is  assumed 


E{n(x 

,y)} 

=  zzn(x 
xy 

,y) 

=  zzn(x 
yx 

.y) 

=  Zn(x, 

X 

y) 

=  Zn(x, 
y 

y) 

(27) 
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and  substituted  into  equations  (26),  then  the  expected  centroid  is 


EExH  -  ExZn 


^^^'    EIH  +  ZIn 
xy    xy 


ZZxH  +  E(n}Zx 
xy 


ZZyH  +  E(n}Zy 


E{Y} 


yx 


ZZH  +  sini 
yx 


Since  E|n}=0,  equations  (28)  become 


e{x: 


ZZxH 
xy 


E{Y} 


ZZH 

xy 

EZyH 

yx 

ZZH 

yx 


(28) 


(29) 


In  general,  the  summations  in  equations  (2?)  are  not  equal  to 
zero,  which  introduces  random  error.   But  if  Zn  =  0,  then  X  =  EJx}, 

y 

and  if  Zq  =  0,  then  Y  =  e{y}.   Thus,  the  summation  process  used  to 

x 
compute  the  centroids  greatly  reduces,  although  in  general  it  does 

not  eliminate,  the  effect  of  noise. 
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To  understand  better  how  noise,  spot  diameter,  and  number  of 
gray  levels  interact  to  affect  spatial  resolution,  consider  the  one- 
dimension  model  in  Figure  10.  This  model  is  essentially  a  column  in 
the  two-dimensional  image  (see  Figure  8)  with  frame  edges  at  x=T  and 
x=B.  For  simplicity,  edge  pixels  are  always  located  at  x=T+2  and 
x=B-2.  (The  extension  to  the  general  two-dimension  case  is  straight- 
forward.) Similar  to  equation  (26),  the  centroid  calculation  is 


X  = 


ZxH(x)  +  Exn(x) 

2C X 

ZH(x)  +  Zn(x) 

X        X 


(30) 


Figure   10.      One-dimension  mode] 


Assume 
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H(x)  = 


K„H^  x=T+2 
T  C 

k:„H  x=B-2 
B  C 

H^  T+3<x<3-3 

0  otherwise 


KH  <H 

—  c—  m 

0<K^<1 

-  T- 
0<<g<1 


and  T<6<B 

where  H^,  a  constant  over  the  line  length,  is  the  gray  level  differ- 
ence and  H^  is  the  maximum  possible  gray  level  difference  (contrast) 
between  black  and  white. 

The  influence  of  noise  and  the  edge  pixels  can  be  described  in 

terms  of  resultants.   The  noise  resultant,  6n^.  is  the  product  of 

random   variables  6  and  n  .  i.e.,  Exn(x)=6n  .  and  Zn(x)=n  .   Dimen- 

•-  Co 

X  X 

sionless  parameter  <^   is  the  fraction  of  the  x=T+2  edge  pixel  covered 

ty  the  line;  and  <^H^  is  the  resultant.    Assume  the  same  for  < 

B 

and  KgH^  at  edge  x=B-2.  During  digitization,  <^  and  <g  are  discre- 
tized  (see  Figure  11)  to  <^  and  <^,  respectively.  For  a  given  line 
segment  at  a  specified  location,  k^  and  k^  are  constants.  The  sub- 
stitution of  these  values  into  equation  (30)  yields 

B-3 

Z      xH  +H  [k  (T+2)+K  (B-2)]+6n 

Y  -  x=T^3  °   '^  ^       B        c 
X  =       — ^ . 

Z     H(x)+H  (k  +<  )+n 
If  x=T+3      c  T  B   c 

B-3 

I  X  =  (T+3)+(T+ii)  +  (T+5)+  ...  +(B-5)+(B-4)+(B-3) 
x=T+3 
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is  added  to 


3-3 
Z  X  =  (B-3)+(B-4)+(B-5)+  ...  +(T+5)+(T+4)+(T+3) 
x=T+3 


then 


B-3 

2   E   X  =  (T+B)+(T+B)+(T+B)+  ...  +(T+B) +(T+B) +(T+B) 
x=T+3 


and 


B-3 
Z  x  =  |(T+B) 
x=T+3 


where  n  is  the  number  of  terms  in  the  sum.  Since  B-3>T+3  and  n>l , 


and 


n  =  (B-3)  -  (T+3)  +  1 


n  =  B-T-5. 


Therefore, 


K^   V  -  (B-T-5)  (T^B) 
x=T+3 


and 


B-3 
Z     H(x)  =  (B-T-5)H 

x=T+3  ^ 
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Subsequently,  equation  (30)  becomes 


X  = 


(31) 


After  dividing  the  top  and  bottom  by  H   equation  (31)  b 


ecomes 


(B-T-5)(T+B)+2C65+<„(T+2)+<„(B-2)] 


2C(B-T-5)+C+K^+Kg] 


(32) 


where  ^  =  —   .       Note,      the  C  ratio     is     similar     to     the     i 

c 
Pratt's  [66]  signal-to-noise  ratio, 


nverse     of 


H     2 

SNR  =    {—) 
0 


(33) 


1.0 


1.0 


Fraction  of  Edge  Pixel  covered 


Figure  11.  The  discretization  of  <  and  k    An  increase  in  H^  im- 
proves spatial  resolution  of  the  edges.  ^ 
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After  studying  Figure  10  and  equation  (32),  it  is  observed 
that  6=T  or  6=B  gives  the  most  noise  error  while 


(B-T-5)(T+B)+2[<„(T+2)+K_(B-2)] 
^    '  2[(B-T-5)+<^*<g]  ^^  ' 


gives  none. 

If  a  line  length  of  d=B-T-5  is  assumed,  equation  (32)  becomes 


(d)(d+2T-^5)+2[65+<„(T+2)+K^(d+T+3)] 
Y  =  L 5 (35) 

^  2[dH+<T^^B^ 


If  5=0, 


(d)(d+2T+5)+2[K„(T+2)+<„(d+T+3)3 

2[d+K^+KgJ 


which  is  the  "exact"  centroid.   If  T  =  0  is  assumed  and  k'  and  k'  are 

i  D 

substituted  into  equation  (35),  the  resolution  error  is 


(d)(d+5)+2[2K^+<g(d+3)]   (d)(d+5)+2C65+2<^+K^(d+3)] 
^e  ^       2[d+K^+<g]  2Cd+5+K^+<^]  ^^^^ 


The  general  relationship  between  E,,  d,  and  R  is  clearer  if 
equation  (37)  is  simplified.  To  assume  the  line  position  is  a  random 
variable  implies  the  following  expected  values: 


e{k^}  =  EJKg}  =  £{<•  }  =  £{<•}  =  ^ 


After  some  algebra,  equation  (37)  becomes 
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J,  _  C(d+5-25) 
e  ~  2(cl+C+1) 


(38) 


ear 


For  c  much  smaller  than  d,  equation  (38)  suggests  an  almost  lin 
relation  between  C  and  R^.   A  spatial  dependency  (perhaps  difficult 
to  verify  experimentally)  is  also  indicated. 


Figure  12.   Icon  rotation  measured  from  the  vertical. 


Icon  rotation.   Next,  the  icon's  angle  of  rotation  is  computed 
(see  Figure  12)  using  the  second  moment  of  inertia.   The  analysis  is 


as  follows; 
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I       =  ZE  I'(x,y)   d^ 
XX  y 

xy 


I       =   EZ   I'(x,y)    d^  (39) 

^^       xy 


P       =  EE   I'(x,y)    d  d 
xy  -^        X  y 

^       xy  ■^ 


where  I  =  moment  of   inertia  about   icon's  x-axis 

I  =  moment  of  inertia  about   icon's  y-axis 

Pyy  =  product  of  inertia  about  icon's  centroid 
xy 

d  =  x  distance  from  pixel  to  icon  centroid 

d  =  y  distance  from  pixel  to  icon  centroid. 
Icon  rotation  is 


2P 
=  ^Tan"^^  ^y  )  (40) 

yy  XX 


The  above  equation  gives  the  angle  to  within  a  constant.  For 
example,  the  analysis  gives  the  same  angle  for  both  icon  orientations 
in  Figure  13-  The  difference  is  resolved  by  constructing  line  "A-A" 
through  the  icon  centroid  and  computing  the  centroids  of  the  upper 
and  lower  segments.  Line  "A-A"  has  the  form 


y.  =  mx  +b 
•'A     A 


where  x  and  y  are  points  on  "A-A"  with 


m  =  -TAN(e) 
and 


b  =  y^   -mx^ 

Icon   Icon 
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Icon 


Upper 
Segment 


Figure  13.  Icon  rotation  is  determined  to  within  a  constant  by  equa- 
tion (40).  ^ 


The  centroid  for  the  upper  segment  is 


^A   ^A 


X..  = 


S   ^      xl'(x,y) 


^A  ^A 


^        ^      I'(x,y) 
x=T  y=L 


\     h 


^        ^      yl'(x,y) 
x=T  y=L 


\      h 


^       ^     I'(x,y) 
x=T  y=L 


(41 
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and  for   the  lower  segment, 


B         R 

E  E       xl'(x,y) 


,     -^A-'. 


L  B  R 

I  I     I'(x,y) 


"="a  ^-^a 


B  R 

I         E       yl'Cx.y) 


(42) 


-      ^   ^=^   y^^A 


L  B  R 

E  E        I'(x,y) 


"=^  ^-h 


The  distances  from  the  icon  centroid  to  the  segment  centroids  are 


2 


d„  =  a/(X,-X^    )^+(Y,-Y^    )' 
U    'V   U  Icon     U  Icon 

(43) 


d,  =  /»/(X-X^    )^+(Y  -Y^    )^ 
L    ^   L  Icon     L  Icon 


If  d,,>d,  ,  e  computed  by  equation  (40)  is  correct;  otherwise  6  must  be 

U   L 

shifted  by 


0-t:    e>o 
Q   =     <  ~  (44) 

)  +  i;    8<0 


Thus,  a  maximum  rotation  of  one  revolution  is  measurable. 
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Information  saved  during  feature  extraction  is,  of  course,  frame 
area,  the  centroid  coordinates  of  each  spot,  the  number  of  spots, 
which  spot  is  the  icon,  and  icon  rotation.  The  areas  and  the  cen- 
troids  are  stored  as  linear  arrays  in  the  order  the  spots  were 
found.  Lastly,  the  problem  of  sorting  this  raw  information  and  using 
it  to  match  spots  in  the  before-  and  after-motion  images  is  solved 
using  pattern  recognition  principles. 
II. 2. 6  Known  Patterns 

In  this  study,  patterns  are  classified  as  either  "known"  or 
"unknown."  Known  patterns  contain  structural  features  (natural, 
artifact,  or  both)  that  are  recognizable  before  and  after  motion. 
Unknown  patterns  are  patterns  which  never  have  recognizable  structure 
or  those  in  which  features  are  obscured  beyond  recognition.  Known 
patterns  containing  only  artifacts  are  considered  here. 

Artifacts  and  their  arrangement  are  selected  to  improve  segmen- 
tation, feature  extraction,  mapping,  and  subsequent  numerical 
analysis.  Examples  of  simple,  easily  analyzed  patterns  are  shown  in 
Figure  14.  These  patterns  are  fine  for  small  rotations  and  are 
easily  recognized  because  the  relative  order  of  the  spots  does  not 
change,  e.g.,  the  upper  left  spot  remains  in  the  same  position-- 
relative  to  the  other  spots — during  motion.  If  rotation  is  45 
degrees  or  more  for  the  square  patterns  and  90  degrees  or  more  for 
the  others,  the  relative  order  is  lost.  The  patterns  in  Figure  15 
allow  spot  identification  after  rotations  up  to  one  revolution.  Once 
the  patterns   in  the  undeformed  and  deformed  images  have  been 
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Figure  14.  Patterns  that  are  easily  recognized  and  analyzed, 
are  for  reference  and  not  part  of  the  pattern. 


Frames 


«  •  • 


•  •  • 

•  •  • 


Figure  15.  Patterns  of  Figure  14  with  icons, 
for  convenience. 


The  icons  are  located 
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recognized,  subsequent  analysis  is  simplified  (especially  for  the 
Lagrangian  description)  because  the  undeformed  spacing  is  approxi- 
mately constant. 
II-2.7  Syntactic  Pattern  Recognition 

Pattern  recognition  is  usually  divided  into  two  classes:  deci- 
sion-theoretic and  syntactic.  In  the  first  class,  a  pattern  is 
represented  by  feature  vectors  and  recognized  by  a  stochastic  parti- 
tioning of  feature  space  [67-69].  Syntactic  recognition  uses  pattern 
structure  and  linguistic  rules  for  analysis  [70,71], 

Syntactic  recognition  assumes  a  pattern  is  described  by  a 
language  which  contains  valid  sentences  composed  from  a  finite  alpha- 
bet or  set  of  symbols.  The  language  is  governed  by  "pattern  grammar" 
or  set  of  grammar  rules.  In  context-free  languages,  pattern  grammar 
is  unrestricted.  Context-sensitive  languages  restrict  pattern 
grammar  on  the  number  of  sentences,  sentence  construction,  or  both. 

Here,  the  pattern  language  is  context-sensitive  and  has  only  one 
sentence  which  is  composed  of  a  fixed  number  of  ordered  symbols.  The 
alphabet  symbols  are  the  spots'  relative  positions  whose  order  is 
determined  in  the  undeformed  image.  A  spot's  context  is  determined 
by  its  0,2,4,6  (or  alternatively  by  0  through  7)  neighbors.  For 
example,  (2,2)  has  neighbors  (1,2),  (2,1),  (3.2),  and  (2,3).  An 
attempt  to  map  (2,2)  in  the  undeformed  image  to,  say  (1,1),  in  the 
deformed  is  "out  of  context"  with  the  neighbors  (see  Figure  16). 
Thus,  correct  pattern  mapping  is  assured  if  every  spot  maps  in  con- 
text with  its  neighbors. 
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(l.n  (1.2)  (1,3)  (1,4)  (1,5) 

(2.1)  (2,2)  (2,3)  (2,4)  (2,5) 

(3.1)  (3,2)  (3,3)  (3,4)  (3,5) 

(4.1)  (4.2)  (4,3)  (4,4)  (4,5) 

(5,1)  (5,2)  (5,3)  (5,4)  (5,5) 


Pattern 


One-sentence  Language 


Figure  16,  A  pattern  and  its  context-sensitive  language.   Note  the 
alphabet  (symbols)  an(i  context  in  the  language. 


Higher  dimensional  pattern  grammars  [70-72]  provide  the  capabil- 
ity to  formally  describe,  recognize,  and  map  patterns  similar  to 
those  in  Figure  15.  Since  a  record  of  each  spot  and  its  neighbors 
must  be  maintained  and  manipulated,  generating  and  parsing  formal 
grammar  for  large  patterns  becomes  an  intense  process.  If  possible, 
trading  some  of  the  formal  elegance  for  an  economic  approximation  is 
highly  desirable. 
^1-2.3  Approximation  of  Syntactic  Pattern  Mapping 

Fortunately  for  many  motions,  spot  relative  positions  change 
very  little,   which  allows   context   to  be  determined  from  spot 


^3 


"addresses."  If  both  images  are  carefully  divided  into  a  number  of 
addresses  (subregions) ,  then  a  spot  has  the  same  address  before  and 
after  motion— or  nearly  so.  The  determination  of  a  spot's  address  is 
a  simple  calculation  involving  centroid  coordinates,  and  recognition 
is  reduced  to  merely  arranging  the  spot  addresses  in  proper  order. 
Thus,  context  is  recognized. 

"Offsets"  are  the  spot  addresses  relative  to  the  icon  address. 
All  offsets  are  defined  according  to  an  icon  address  of  0  (see  Figure 
17). 


Image 
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Figure  17.  Image  addresses  and  offsets, 
image  edges. 


Some  addresses  overlao 
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Pattern  geometry  affects  the  upper  limits  of  measurable  strain 
because  large  strain  displacements  may  change  the  order  of  the  off- 
sets in  the  deformed  image.  Spreading  spots  further  apart  (in  the 
undeformed  image)  increases  the  size  of  a  spot's  subregion.  A  larger 
subregion  allows  the  spot  more  strain  displacement  before  any  change 
in  offset  order  occurs. 

The  details  are  best  illustrated  by  example  (see  Figure  18).  It 
is  assumed  that  the  undeformed  image  is  analyzed  first  and,  with  one 
exception,   both   images  receive  the  same  preliminary  analysis. 


Undeformed 


Deformed 


Figure  18.  A  simple  example  of  motion  Ou. /9X .=0.05,  u=v=-10  pixels, 
9=120°,  E^. =0.0525).  The  origin  "^is  in  the  upper  left 
corner  of  each  image. 
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Successful  mapping  is  initially  assumed  and  later  verified  by  four 
final  checks. 

First,  the  image  is  segmented  and  the  features  are  extracted. 
Scanning  is  by  column  and  the  spot  centroids  are  stored  in  the  order 
they  were  found.  (See  columns  marked  "Raw"  in  Tables  1  and  2.)  The 
icon  is  identified,  and  it's  rotation  is  also  computed. 


Table   1  . 

Unordered  spo 

t  coordinates   in   the  und 

eformed   imac 

!e. 

Raw 

Refe 

rence 

From 

Icon 

X 

Y 

X 

Y 

X 

Y 

58.0101 

57.9660 

57.4541 

57.8310 

-70.5459 

-70.1690 

93.0079 

58.0056 

92.4517 

57.9438 

-35.5483 

-70.0562 

163.0518 

57.9950 

162.4955 

58.0797 

34.4955 

-69.9203 

197.9896 

58.0041 

197.4332 

58.1617 

69.4332 

-69.8383 

127.9832 

57.9801 

127.4271 

57.9914 

-0.5729 

-70.0086 

58.0481 

93.0576 

57.4187 

92.9227 

-70.5813 

-35.0773 

93.0216 

92.9791 

92.3923 

92.9173 

-35.6077 

-35.0827 

197.9616 

93.0132 

197.3321 

93.1757 

69.3321 

-34.8243 

127.9396 

92.9724 

127.3103 

92.9836 

-0.6897 

-35.0164 

163.0107 

92.9915 

162.3813 

93.0760 

34.3813 

-34.9240 

128,7025 

127.9873 

128.0000 

128.0000 

0.0000 

0.0000 

57.9599 

128.0263 

57.2575 

127.8911 

-70.7425 

-0.1039 

93.0284 

128.0002 

92.3260 

127.9383 

-35.6740 

-0.0617 

198.0660 

128.0575 

197.3632 

128.2151 

69.3632 

0.2151 

63.0010 

127.9894 

162.2984 

128.0738 

34.2984 

0.0738 

92.9964 

163.0297 

92.2207 

162.9676 

-35.7793 

34.9676 

127.9581 

162.9798 

127.1824 

162.9909 

-0.8176 

34.9909 

162.9653 

163.0147 

162.1896 

163.0989 

34.1896 

35.0989 

197.9768 

162.9531 

197.2011 

163.1105 

69.2011 

34.1105 

57.9856 

162.9875 

57.2101 

162.8523 

-70.7899 

34.8523 

58.06  35 

198.0315 

57.2148 

197.8964 

-70.7852 

69.8964 

92.9969 

193.0169 

92.1481 

197.9548 

-35.8519 

69.9548 

128.0099 

198.0273 

127.1610 

198.0384 

-0.8390 

70.0384 

163.0027 

198.0136 

162.1538 

198.0978 

34.1538 

70.0973 

197.9936 

197.9982 

197.1446 

198.1556 

69.1446 

70.1556 

NOTE:   Units  are  pixels.   "Raw"  coordinates  are  the  spot  centroids 
in  the  image.   "Reference"  and  "From  Icon"  values  are  computed  by 
equations  (46)  and  (47),  respectively.  See  Figure  18. 
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Table  2.  Unordered  spot  coordinates  in  the  deformed  image, 


Raw Reference 


From  Icon 


98.5336  33.4158  51.9634  192.1880         -76. 0366  64   1880 

^ll'lll"^  50.2682  53.0816         155.3833         -74.9184  27!3833 

1AII\  ^r1-V.ll  ?^-^532         196.7857         -39.5468  68.7857 

9.3470 
32.0309 


163.9343  67.1092  54.2228  118.6530  -73.7772 

111.3736  81.1960  89.5630  I6O.0309         -38   4370 

196.6288  84.0388  55.4298  81.8551  -72.5702  -46."l'449 

73.5070 
4.7419 


58.7113  95.2677  124.9335  201.5070  -3. 0665 

144.0612  98.0816  90.7333  123.2581  -37.2667 

229.4016  100.8559  56.5016  45.0350  -71.4984  -82'9650 

91.4833  112.1065  126.0252  164.6782  -1.9748  36'6782 

175.7262  114.9435  91.8919  86.5161  -36.IO8I  -4r4839 

123.7393  129.6776  128.0000  128.0000  0.0000  0  0000 

38.9059  126.2522  161.4258  206.0458  33.4258  78'o453 

209.4617  131.8162  93.0300  49.7056  -34.9700  -78    2944 

71.5500  143.1225  162.6010  169.3190  34.6010  41 '3190 

156.8817  145.8710  128.3497  91.1147  0.3497  -36  8853 

18.9873  157.2419  197.9716  210.6846  69.9715  82"6846 

104.3150  160.0107  163.7403  132.4753  35.7403  4   4753 

189.5733  162.8025  129.5597  54.3186  1.5597  -73*6814 

51.6854  174.0585  199.0750  173.9321  71.0750  45."9321 

136.9841  176.8253  164.8545  95.7498  36.8545  -32  2502 

84.3153  190.9335  200.2602  137.2151  72.2602  9*2151 

109.6751  193.7303  166.0404  58.9647  38.0404  -59  0353 

117.1079  207.7552  201.3280  100.3761  73.3280  -?7'5239 

149.7807  224.6135  202.4800  53.6284  74.4800  -64'37i6 


NOTE:      Units  are  pixels. "Raw"   coordinates  are  the  spot  centroids 
m  the  image.      "Reference"  and  "From  Icon"  values  are  computed  by 
equations    (46)   and   (47),    respectively.      See  Figure   18 


All    spots   are    moved    into    a    "reference   position"    by   translation 
and  rotation   (see  Figure   19).      Translation  is  accomplished  by  adding 
X^   to  the  X  and  Y     to  the  Y  of  each  spot;    X     and  Y     are 


X^   =   X     -X^ 
T  cp      Icon 


Y^   =   Y     -Y 
T  cp      Icon 


(45) 
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where  X^^  and  Y^^  are  the  coordinates  of  the  icon  in  reference  posi- 
tion (typically  the  image  center).  Rotation  for  each  spot  is 
computed  from  the  x  and  y  distance  between  it  and  the  icon.  The 
computation  is 


X  =  x-x, 

p      Icon 


Y  =  Y-Y, 
p      Icon 


v> 


'-  ^'i'^l 


_T  Y 
co^  =  Tan  \^) 


^Rof  "  X+p[C0S(u)  -9)-C0S  0)  ]+X 

nei  z  7. 


z"      T 


(46) 


Yo^f.  =   Y  +  p[sin(tj  -e)-sin   u  ]+Y 
nei  z  7. 


z"   T 


Where  X^^^^  and  Y^^^^  are  the  new  coordinates  of  each  spot  in  the 
reference  position.  The  results  of  translation  and  rotation  are 
shown  in  Tables  1  and  2  under  the  "Reference"  column. 

Position  relative  to  the  icon,  i.e.,  distance  from  the  icon,  is 


^Rel   ^Ref   ^Icon 


Y    =  Y    -  Y 
Rel    Ref    Icon 


(47) 


48 


Deformed  Image 


Reference  Position 


Figure  19.  The  deformed  image  and  its  reference  position. 


The  distances  from  the  icon  are  shown  in  Tables  1  and  2  under  the 
columns  marked  "From  Icon."  Although  all  spots  have  been  located 
relative  to  the  icon,  recognizing  the  pattern  they  form  and  arranging 
them  in  proper  order  (by  address)  is  not  always  easy-especially  if 
rotation  is  more  than  a  few  degrees. 

Address  computation  is  based  on  "learning"  the  undeformed 
image.  The  spots  nearest  and  farthest  from  the  icon  row  and  column 
are  located  in  the  undeformed  image  and  used  to  calculate  the  address 
"offset,"  ^Qff^Q^,    by  the  following: 
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X    =  MAX(X) 
max 


X  .   =  .MIN(X) 
mm 


Y    =  MAX(Y) 
max 


Y  .   =  MIN(Y) 
min 


N  =  NINKtt^) 
X        X  . 
rain 


N  =  NINTCy^^) 
min 


X   -N  (X  .  ) 
X     =  Y    +  niax  X  min 
space    min      N  +1 


Y   -N  (Y  .  ) 
Y      =  Y   +  '"ax  y  min 
space    min     N  +1 

y 


(48) 


space 


(49) 


X  Y 

Vfset  =  NINT(3^)  .  NINT(^-^^)(N3^.^^) 


space 


space 


(50) 


where  X^^^^^  =  maximum  X  centroid 


^rain  ^  minimum  X  centroid 


^max  ^  maximum  Y  centroid 


^min  ^  minimum  Y  centroid 


N„  =  number  of  rows  above  or  below  icon 


:;_  1*1- 
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Ny  =  number  of  columns  left  or  right  of  icon 

^rows  "  number  of  rows  in  the  image 

^space  "  approximate  row  spacing 

^space  "  approximate  column  spacing 

'^ shift  ^   ni^niber  of  possible  columns  in  image. 

•Function  MINT  rounds  numbers  to  the  nearest  integer  value.   Values 

^space'  ^space'  ^"'^  ^shift  ^^^    saved  from  the  undeforraed  image  and 
used  in  the  deformed. 

Fortunately,  arranging  the  spots  in  proper  sequence  is  a  simple 

matter  of  bubble  sorting  [73]  the  offsets  in  ascending  order.   By 

equation  (50),  offsets  (see  column  "Offset"  in  Tables  3  and  4) 

increase  down  columns  and  left  to  right.   Column  "Key"  is  the  order 

in  which  the  spots  were  found.    Note,  the  two-dimension  pattern 

context  is  now  represented  by  a  one-dimension  array,  X   ,  which 

Rel 

corresponds  to  the  ordered  spot  addresses. 

Recognition  of  the  two-dimension  context  is  accomplished  by 
determining  the  row-column  order  of  the  spots.  The  left-most  column 
is  assumed  to  begin  with  the  lowest  offset  and  continue  with  ascend- 
ing offset  until  the  X  coordinate  decreases.  Column  2  begins  with 
the  next  highest  offset  and  continues  until  the  X  coordinate  again 
decreases.  The  process  continues  until  the  offset  array  is  exhaust- 
ed. By  incrementing  row  and  column  counters,  the  relative  spot  order 
is  determined  which  establishes  context  (see  Tables  5-8). 

Until  now,  both  undeformed  and  deformed  images  were  processed 
the  same  (except  for  learning  the  undeformed  image)  and  assumed  to 
have  the  same  context.   This  assumption  is  verified  by  the  following 


Table  3.  Ordered  spot  coordinates  in  the  undeformed  image. 
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Key 
1 

Refe 

rence 

From 

Icon 

Offset 

X 

Y 

X 

Y 

-18 

57.4541 

57.8310 

-70.5459 

-70.1690 

-17 

2 

92.4517 

57.9438 

-35.5483 

-70.0562 

-16 

5 

127.4271 

57.9914 

-0.5729 

-70.0086 

-15 

3 

162.4955 

58.0797 

34.4955 

-69.9203 

-14 

4 

197.4332 

58.1617 

69.4332 

-59.8383 

-10 

6 

57.4187 

92.9227 

-70.5813 

-35.0773 

-9 

7 

92.3923 

92.9173 

-35.6077 

-35.0827 

-8 

9 

127.3103 

92.9836 

-0.6897 

-35.0164 

-7 

10 

162.3813 

93.0760 

34.3813 

-34.9240 

-6 

8 

197.3321 

93.1757 

69.3321 

-34.8243 

-2 

12 

57.2575 

127.8911 

-70.7425 

-0.1089 

-1 

13 

92.3260 

127.9383 

-35.6740 

-0.0617 

0 

n 

128.0000 

128.0000 

0.0000 

0.0000 

1 

15 

162.2984 

128.0738 

34.2984 

0.0738 

2 

14 

197.3632 

128.2151 

69.3632 

0.2151 

6 

20 

57.2101 

162.8523 

-70.7899 

34.8523 

7 

16 

92.2207 

162.9676 

-35.7793 

34.9676 

8 

17 

127.1824 

162.9909 

-0.8176 

34.9909 

9 

18 

162.1896 

163.0989 

34.1896 

35.0989 

10 

19 

197.2011 

163.1 105 

69.2011 

35.1105 

14 

21 

57.2143 

197.8964 

-70.7852 

69.8964 

15 

22 

92.1481 

197.9548 

-35.3519 

69.9548 

16 

23 

127.1610 

198.0384 

-0.8390 

70.0384 

17 

24 

162.1538 

198.0978 

34.1538 

70.0978 

18 

\Tr\mrt                    TT 

25 

197.1446 

198.1556 

69.1446 

70.1556 

Units  are  pixels.  Offsets  are  computed  by  equation  (50) 
Keys  are  the  order  in  which  the  spots  were  found.  Undeformed 
spacing  is  35  pixels  in  x  and  y.  See  Figure  18. 
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Table  k.     Ordered  spot  coordinates  in  the  deformed  image. 


Key 
9 

Refe 

rence 

From 

Icon 

Offset 

X 

Y 

X 

Y 

-18 

56.5016 

45.0350 

-71 .4984 

-82.9650 

-17 

U 

93.0300 

49.7056 

-34.9700 

-78.2944 

-16 

19 

129.5597 

54.3186 

1.5597 

-73.6814 

-15 

23 

166.0404 

58.9647 

38.0404 

-69.0353 

-14 

25 

202.4800 

63.6284 

74.4800 

-64.3716 

-10 

6 

55.4298 

81  .8551 

-72.5702 

-46.1449 

-9 

11 

91.8919 

86.5161 

-36.1081 

-41.4839 

-8 

16 

128.3497 

91  .1147 

0.3497 

-36.8853 

-7 

21 

164.8545 

95.7498 

36.8545 

-32.2502 

-6 

24 

201  .3280 

100.3761 

73.3280 

-27.6239 

-2 

4 

54.2228 

118.6530 

-73.7772 

-9.3470 

-1 

8 

90.7333 

123.2581 

-37.2667 

-4.7419 

0 

12 

128.0000 

128.0000 

0.0000 

0.0000 

1 

18 

163.7403 

132.4753 

35.7403 

4.4753 

2 

22 

200.2602 

137.2151 

72.2602 

9.2151 

6 

2 

53.0816 

155.3833 

-74.9184 

27.3833 

7 

5 

89.5630 

160.0309 

-38.4370 

32.0309 

8 

10 

126.0252 

164.6782 

-1  .9748 

36.6782 

9 

15 

162.6010 

169.3190 

34.6010 

41.3190 

10 

20 

199.0750 

173.9321 

71  .0750 

45.9321 

14 

1 

51.9634 

192.1880 

-76.0366 

64.1880 

15 

3 

88.4532 

196.7857 

-39.5468 

68.7857 

16 

7 

124.9335 

201.5070 

-3.0665 

73.5070 

17 

13 

161  .4258 

206.0458 

33.4258 

78.0458 

18 

17 

197.9716 

210.6846 

69.9716 

82.6846 

NOTE:      Units  are  pixels.      Offsets  are  computed  by  equation   (50) 
Keys  are  the  order   in  which   the  spots  were  found.     See  Figure   18. 
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Table  5.      Spot  distance   from  origin   in  undeformed  image. 


ROW  1         X: 
Y: 


ROW  2         X: 


ROW  5         X: 


COL  1 


58.0101   58.0481   57.9599   57.9856   58.0635 
57.9660   93.0576  128.0263  162.9875   198.0315 


COL  1 


93.0079       93.0216        93.0284       92.9964       92.9969 
Y:  58.0056       92.9791      128.0002      163.0297      198.0159 

COL  1  2  ?  U  ^ 


f^OW  3  X:  127.9832     127.9396      128.7025      127.9581      128.0099 

Y:  57.9801        92.9724     127.9873      162.9798      198.0273 

COL  1  2  ^  li  :; 


f^OW  4  X:  163.0518      163.0107      I63.OOIO     162.9653      163.0027 

Y:  57.9950       92.9915     127.9894      163.0147      198.0136 

COL  1  2  ^  a  . 


197.9896     197.9616     198.0660     197. 9763     197.9936 
Y:  58.0041        93.0182     128.0575      162.9531      197.9982 


NOTE:      Units  are  pixels.     The  origin   is   the  upper  left  corner  of   the 
image.     See  Figure   18. 


Table  6.      Spot  distance   from  origin  in  deformed  image, 


COL 


ROW   1  X: 


ROW   4  X; 


ROW  5  X: 


ROW   2  X: 


ROW   3  X; 
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229.4016      196.6288      16-3.9343      131.2724       98.5336 
Y:  100.8559        84.0388        67.1092       50.2682       33.4158 

COL  1  2  ^       ■  n  = 


209.4617      176.7262      144.0612      111.3736       78.6761 
^:  131.8162      114.9435       98.0816        81.1960       64.3727 

COL  1  2  9  11  c. 


189.5733     156.8817      123.7393       91.4833       58.7113 
Y:  162.8025      145.8710      129.6776      112.1065       95.2677 

COL  1  2  ^  n  c 


169.6761      136.9841      104.3150       71.5500        38.9059 
Y:  193.7303      176.8253      160.0107      143.1225      126.2522 

COL  1  2  ?  a  K 


149.7307      117.1079        84.3163       51.6854        l8   9873 
Y:  224.6135      207.7552      190.9335      174.0585      157.2419 


NOTE:      Units  are  pixels"!     The  origin   is   the  upper  left  corner  of   the 
image.     See  Figure   18. 
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Table  7.  Spot  distance  from  icon  in  undeforraed  image. 


COL  1 


ROW   1  X:  -70.5459      -70. 5813     "70,7425     "70.7899     "70.7852 

Y:  -70.1690     -35.0773       "0.1089       34.8523       69.8954 

COL  1  2  ^  U  ^ 


ROW  2        X:     -35.5483  -35.6077  -35.6740  -35.7793  "35.8519 
Y:     -70.0562  "35.0827   "0.0617   34.9676   69.9548 

COL  1         2         ?         U        c: 


ROW  3  X:  "0.5729        -0.6897         0.0000       "0.8175       "0.8390 

Y:  "70.0086     "35.0164         0.0000       34.9909       70.0384 

COL  1  2  ?  4  ^ 


ROW  4  X:  34.4955       34.3813       34.2984       34.1896       34.1538 

Y:  -69.9203     "34.9240         0.0738       35.0989       70.0978 

COL  1  2  ^  ii  ^ 


ROW  5        X:      69.4332   69.3321   69.3632   59.2011   69.1446 
Y:     "69.8383  "34.8243    0.2151   35.1105   70.1556 


NOTE:   Units  are  pixels.  Spacing  is  35  pixels  in  x  and  y.  See 
Figure  18. 


:oL 


COL 
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Table  8.      Spot  distance  from   icon   in  deformed   image, 


ROW    1  X: 


ROW  2  X: 


ROW   J4  X: 


COL  1 


-71.4984     -72.5702     -73.7772     -74.9184     -76.0368 
Y:  -82.9650     -46.1449       -9.3470       27.3833       64.1880 


-34.9700     -36.1081      -37.2667     -38.4370     -39.5468 
Y:  -78.2944     -41.4839       -4.7419       32.0309       68.7857 


ROW  3  X:  1.5597         0.3497         0.0000       -1.9748       -3. 0665 

Y:  -73.6814     -36.8853         0.0000       36.6782       73.5070 

COL  1  2  ^  a  ^ 


38.0404       36.8545       35.7403       34.6010       33.4258 
Y:  -69.0353     -32.2502  4.4753       41.3190       78.0458 

COL  \  2  ^  h  ^ 


ROW  5        X:      74.4800   73-3280   72.2602   71.0750   69.9716 
Y:     -64.3716  -27.6239    9.2151   45.9321   82.6845 


NOTE:   Units  are  pixels.  See  Figure  18. 
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checks: 

1.  same  number  of  rows  in  each  image 

2.  same  number  of  columns  in  each  image 

3.  same  number  of  spots  in  each  image 

4.  rows  times  columns  equal  number  of  spots. 

Of  course,  if  any  check  fails,  the  images  are  "out  of  con- 
text." If  strains  are  large  enough,  context  similarities  between 
images  cannot  be  recognized  and  an  error  message  is  printed.  Assum- 
ing large,  local  rotation  does  not  occur  in  the  material,  recognizing 
similar  context  between  the  two  patterns  that  does  not  exist  is  very 
remote.  Thus  if  no  check  fails,  the  approximation  algorithm  estab- 
lishes a  one-to-one  relationship,  or  "context  mapping,"  between  spots 
in  the  two  images. 


CHAPTER  III 
IMAGE  GENERATION  AND  ANALYSIS 


III.1   Synthetic  Images 

Computer  generated  images,  or  "synthetic"  images,  were  used  as 
the  primary  development  tool  and  to  study  the  influence  of  various 
parameters.  The  options  data  file  shown  in  Table  9  was  used  to  gen- 
erate these  256X256  pixel  images. 

Motion  is  limited  to  affine  transformations  plus  rigid-body 
rotation  about  the  image  center.  The  transformations  are 


X  -  c^X  +  c^Y  +  c 


c,,X  +  c^Y  +  c. 
4     5     0 


(51) 


There  are  three  image  output  options  available.  The  "BINARY" 
option  is  the  machine  representation  of  the  image.  "DECIMAL"  is  the 
machine  version  converted  to  a  readable  form  (see,  for  example, 
Figure  4b).  "PLOT"  displays  a  thresholded  image,  suitable  for  hard 
copy,  on  a  graphics  terminal  (see,  for  example,  Figure  15). 
"U123.DAT,"  "NIMG.DAT,"  "D123.DAT,"  and  "DIMG.DAT"  are  user-defined 
file  names.  The  image  generator  automatically  reads  the  input  file 
and  generates  the  specified  output  thereby  allowing  the  user  a  choice 
of  interactive  or  batch  processing. 
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Table  9.      Input  file  for  synthetic   image  generator. 


IMAGE  GENERATOR  INPUT  DATA 
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SPOT  GEOMETRY 


RADIUS   XSPACING   YSPACING 


4.00 


35.00 


35.00 


PATTERN  SIZE 
XCOLS   YROWS 


GRAY  LEVELS 

BLACK   WHITE 
00000   00200 


DISTRIBUTION  TYPE:   N 


NORMAL 


UNIFORM 


NOISE  DATA 


STANDARD  DEVIATION:   007.000 


IMAGE  MOTION 


MOTION  TYPE: 


1 --POLYNOMIAL  ONLY 
2--P0LYN0MIAL  AND  ROTATION 
3--R0TATI0N  ONLY 

POLYNOMIAL        X 


x:   +1.05000   +0.05000 
y:   +0.05000   +1.05000 

ROTATION  (DEGREES):   +120.000 


TRANSLATION 

-010.0000 
-010.0000 


IMAGE 

OUTPUT 

FORMAT 

UNDEFORMED 

Y/N 

DEFORMED 

Y/N 

BINARY: 

DECIMAL: 

PLOT: 

U123.DAT 
NIMG.DAT 
* SCREEN* 

Y 

N 
N 

D123.DAT 
DIMG.DAT 
*SCREEN» 

Y 
N 
N 
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111.2  Image  Analysis 
Similar  to  image  generation,  image  analysis  begins  with  an  op- 
tions file  which  is  shown  in  Table  10.  If  "USE  THESE  FILE  NAMES?"  is 
answered  no,  the  user  is  automatically  prompted  for  the  file  name. 
Since  no  other  information  is  required  by  the  program,  the  user  is 
free  to  choose  interactive  or  batch  processing. 


Table  10.   Image  analysis  options  file. 


********  IMAGE  ANALYSIS  INPUT/OUTPUT  OPTIONS  ******** 


INPUT  SPECIFICATIONS 


UNDEFORMED    DEFORMED 


IMAGE  FILE  NAME: 

USE  THESE  FILE  NAMES?; 


U123.DAT 
N 


D123.DAT 
N 


IMAGE  OUTPUT 


UNDEFORMED 


DEFORMED 
+ 

Y 
Y 
Y 


UNORDERED  SPOT  DATA: 
DISTANCE  FROM  ORIGIN: 
DISTANCE  FROM  ICON: 


Y 
Y 
Y 


DISPLACEMENT  ANALYSIS 

TOTAL  DISPLACEMENT: 
DISPLACEMENT  GRADIENTS; 
LAGRANGIAN  STRAIN: 
TAYLOR  SERIES  STRAIN: 


OUTPUT 
—  + 

Y 
Y 
Y 
Y 


MOTION  ANALYSIS 


DEFORMATION  GRADIENT 
GREEN  DEFORM.  TENSOR 
RIGHT  STRETCH  TENSOR 
ROTATION  TENSOR: 
LAGRANGIAN  STRAIN: 


OUTPUT 

Y 
Y 
Y 
Y 
Y 
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After  segmentation,  feature  extraction,  and  context  recognition, 
automatic  processing  continues  with  motion  analysis.  The  motion  of 
each  spot  is  resolved  and  the  displacement  is  computed  by  equation 
(3)  (see  Table  11).  Next,  the  deformation  and  displacement  gradients 
at  each  spot  are  computed  by  differentiating  the  functions  used  to 
describe  motion  and  displacement. 

Table  11.  Displacement  of  each  spot. 


COL 


1 


ROW  1 


COL 


X: 
Y: 


171.3916 
42.8899 

1 


138.5808 
-9.0188 


105.9744 
-60.9171 


73.2868 
-112.7193 


40.4701 
■164.6158 


ROW  2 

X: 

Y: 

116.4538 
73.8106 

83.7046 
21  .9643 

51.0328 
-29.9186 

18.3772 
-81 .8337 

-14.3208 
-133.6442 

COL 

1 

2 

3 

4 

5 

ROW  3 

X: 
Y: 

61 .5900 
104.3223 

28.9421 
52.8985 

-4.9632 
1  .6903 

-35.4747 
-50.8733 

-69.2986 
-102.7595 

COL 

1 

2 

3 

4 

5 

ROW   4 


COL 


X: 
Y: 


6.6243 
135.7353 

1 


-26.0267 
83.8338 


-58.6861 
32.0212 


-91 .4154 
-19.8922 


-124.0968 
-71  .7614 


ROW  5 


X: 
Y: 


-48.2088 
166.6094 


-80.8538 
14.7370 


■113.7497 
62.8761 


■146, 
11  , 


2914 
1054 


-179.0063 
-40.7563 


NOTE:      Units  are  pixels.      Compare   images   in  Figure    18.      Displacements 
are  computed  using  the  data   in  Tables  5  and  6    in  equation   (3). 


The     choice     of     a     mapping     model     depends,     of     course,     on     the 
application.  Intuition,      a     priori     knowledge,      elasticity     theory. 
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numerical  efficiency,  etc.  are  used  to  select  the  approximating  func- 
tions. In  this  study,  simple  polynomials,  linear  or  second  order, 
adequately  approximate  the  motion  and  displacement  in  the  synthetic 
and  real  images.  The  models  are  linear 


^  =  °n^^°2i  '   °3i' 


^  =  ^12^^  °22^  V 


u  =  c, ^X  +  c^^  +  c  Y 
13     23    33 


-  =  °UX.  c^,  ^  C3^Y 


and  second  order 


(52; 


(53) 


''   ~-    "ll^'   '    °21^   ^   °31    *   ""h/   '    %l^   *   -^61^' 


y   =   C^2^      ^   <=22^   '   °32   *   %2^     '   Sz"-   ^   %2^^ 


u  =  0^/  .  C23X  *  C33  .  c^3y2  .  C53Y  .  Cg3XY 


V  =   c^^X     .   c^^X   .   03^   .   c^^Y      .   c^^Y   .   Cg^XY 


These  models  are  fitted  to  a  small  neighborhood  of  spots  and 
differentiated   to   obtain   the   displacement   and   deformation 
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gradients.  Typically,  the  neighborhoods  are  3x3,  5X5,  or  7x7  and  the 
gradients  are  computed  for  the  center  spot.  Exceptions  are  the  spots 
near  the  edge  of  the  pattern  and  their  gradients  are  computed  like  a 
forward-backward  difference. 

For  both  models,  the  coefficients  are  determined  by  least 
squares  evaluation.  Based  on  a  modified  form  of  the  Conte  and  Deboor 
[74]  notation,  the  normal  equations  are 


where 


$$  c  =  $f 


(54) 


1    1 

Y    Y 
1     2 


.  1 
.  Y 


Linear 


(55) 


^1^1   V2 


N 
1 


^N^N 


Second 
Order 


6i\ 


'11  •••  "14 


Linear 


'31     "34 


'11  •••  "14 


o1 


'64 


Second 
order 


(56) 


f  = 


^1   ^1   ^1   ^1 


"n  ^N  \     \ 


(57) 


and  N  is  the  number  of  spots  in  the  neighborhood.  Let 


and 


Equations  (54)  become 


T 
H      =   A 


$f  =  Z 


Ac  =  Z 


(58) 


which  are  solvable  by  Gauss-Jordan  elimination. 
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Hornbeck  [75]  points  out  Gauss-Jordan  elimination  may  be  used  to 
solve  equations  (58),  but  unfortunately  the  set  is  extremely 
ill-conditioned.  Taking  his  advice,  double  precision  arithmetic  is 
used  to  cope  with  the  large  variation  in  the  magnitudes  of  the  co- 
efficients in  any  given  row.  In  addition  to  his  suggestions,  (jj  and  f 
are  scaled  before  solution  and  the  c's  are  reconverted  during 
gradient  computation. 

For  a  variety  of  programming  reasons,  solution  of  equations  (58) 
is  not  performed  directly  by  Gauss-Jordan  elimination.  Instead,  the 
inverse  of  A  is  computed  using  Hornbeck' s  [75]  Gauss-Jordan  matrix 
inversion  subroutine,  with  the  addition  of  appropriate  error  traps, 
and  c  is  solved  by 


5  =  A~  ?  (59) 


How  accurate  are  the  values  of  c  and  how  well  do  equations  (52) 
or  (53)  fit  the  data?  Debugging  experience  indicates  the  number  of 
reliable  digits  (numerically  speaking,  not  experimentally)  runs  in 
the  teens.  In  this  study  the  solutions  are  known,  but  if  they  were 
unknown,  goodness-of-fit  and  the  contribution  of  each  term  could  be 
easily  tested  by  the  methods  suggested  by  Miller  and  Freund  [63]. 

The  example  of  Figure  18  is  continued  with  the  calculation  of 
c.   Assume  the  linear  model  of  equation  (52)  and  a  3X3  neighborhood 
centered  at  spot  (3,3).   A  substitution  of  data  from  Table  5  into 
equation  (55)  produces 
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93.0216 

1  .0000 

92.9791 
128.0002 

93.0284 

1.0000 

92.9964 

1  .0000 

163.0297 

127.9396 

1.0000 

92.9724 

128.7025 

1  .0000 

127.9873 

127.9581 

1.0000 

162.9798 

163.0107 

1  .0000 

92.9915 

163.0010 

1.0000 

127.9894 

162.9653 

1  .0000 

163.0147 

(60) 


Likewise,  data  from  Tables  6  and  1 1  in  equation  (57)  lead  to 


f  = 


176.7262 

114.9435 

83.7046 

21  .9643 

144.0612 

98.0816 

51.0328 

-29.9186 

111  .3736 

81  .1960 

18.3772 

-81  .8337 

156.8817 

145.8710 

28.9421 

52.8985 

123.7393 

129.6776 

-4.9632 

1  .6903 

91.4833 

112.1065 

-36.4747 

-50.8733 

136.9841 

176.8253 

-26.0267 

83.8338 

104.3150 

160.0107 

-58.6861 

32.0212 

71  .5500 

143.1225 

-91  .4154 

-19.8922 

(61) 


The  solution  for  c  is 


-0.56849  0.88427 

316.52130  77.50840 

_-0. 93425  -0.48175 


-1.56849 

316.52130 

-0.93425 


0 

.88427 

77 

.50840 

-1 

.48165 

(62) 


The    motion    functions     (i.e.,     the    "mathematical"    maps)    in    equa- 
tions   (52)   become 


X  =   -0.56849X  +   316.52130   -  0.93425Y 


y  =  0.88427X  +  77.50840  -  0.48175Y 


(63) 
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and  the  displacement  maps  in  equations  (52)  be 


come 


u  =  -1.56849X  +  315.52130  -  0.934251 


V  =  0.88427X  +  77.5084  -  1  .48175Y 


(64) 


The  deformation  and  displacement  gradients,  computed  by  equa- 
tions (2)  and  (4),  respectively,  become 


-0.55849 

0.88427 


-0.93425 
-0.48175 


(55) 


J  = 


-1.56849     -0.93425 
0.88427     -1.48175 


(56) 


Strain,  computed  by  equations  (11)  or  (12),  i: 


0.05255      0.05255 
0.05255      0.05246 


(67) 


By  the  Taylor  series  method  of  equation  (13),  strain  u 


0.05124      0.04763 
0.04763      0.05115 


(58) 
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Strain  computed  according  to  small  displacement  theory  is,  in 
this  case,  quite  inaccurate.   By  equations  (14), 


-1  .56849 
-0.02499 


-0.02499 
■1.48175 


(69) 


Rigid-body  rotation  is  computed  as  follows.   By  equation  (8), 
the  Green  deformation  tensor  is 


1  .10509 
0.1051 1 


0.10511 
1.10491 


(70) 


The  eigenvalues  of  C  are 


1  .21009 
0.00000 


0.00000 
0.99990 


(71) 


1  /2 
and  C  ^  is 

~P 


,1/2 


1  .10004 
0.00000 


0.00000 
0.99995 


(72) 


Since  U 
-P 


C   ,  then 
-P 


U  = 


1  .05000 
0.05005 


0.05005 
1.05000 


(73) 
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Rotation  tensor  R,  computed  by  equation  (17),  is 


-0.50016 
0.86596 


-0.86596 
-0.5001 1 


(7iJ) 


The  remainder  of  the  output  for  the  example  in  Figure  18  is 
shown  in  Tables  12  through  19. 
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Table 

12.      Displacement  gr 

adients. 

COL 

1 

2 

3 

4 

5 

ROW   1 

COL 

3u/3X 
3v/9X 
3u/8Y 
3V/3Y' 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

1 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

2 

-1.56849 
0.88427 

-0.93425 
-1  .48175 

3 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

4 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

5 

ROW   2 
COL 

9u/3X 
3v/3X 
3u/3Y 
3V/3Y 

-1  .56849 

0.88427 

-0.93425 

-1  .48175 

1 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

2 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

3 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

4 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

5 

ROW   3 

COL 

3u/3X 
3V/3X 
3u/3Y 
9V/3Y 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

1 

-1.56849 

0.88427 

-0.93425 

-1  .43175 

2 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

3 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

4 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

5 

ROW   4 
COL 

9u/3X 
3v/3X 
3U/3Y 
3V/3Y 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

1 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

2 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

3 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

4 

-1  .56849 

0.88427 

-0.93425 

-1  .48175 

5 

ROW   5 

3U/3X 
3V/3X 
3u/3Y 
3V/3Y 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

-1.56849 

0.88427 

-0.93425 

-1  .48175 

NOTE:   Correct  values  are  3u/3X  =  -I.56830,  3v/3X  =  0.88433, 
3U/3Y  =  -0.93433,  3v/3Y  =  -1.48170. 


Table  13.     Lagranglan  strain  from  displacement  gradients, 


COL 


71 


ROW   1 


;ii 
;i2 

^22 


52554. 

52554. 

52554. 

52554. 

52554 

52557. 

52557. 

52557. 

52557. 

52557 

52459. 

52459. 

52459. 

52459. 

52459 

COL 


ROW  2 


^11 

;i2 

^22 


:               52554. 

52554. 

52554. 

52554. 

52554 

:               52557. 

52557. 

52557. 

52557. 

52557 

:               52459. 

52459. 

52459. 

52459. 

52459 

COL 


ROW  3 


'11 
^22 


:               52554. 

52554. 

52554. 

52554. 

52554 

:               52557. 

52557. 

52557. 

52557. 

52557 

:               52459. 

52459. 

52459. 

52459. 

52459 

COL 


ROW  4 


M2 

'22 


52554. 

52554. 

52554. 

52554. 

52554 

52557. 

52557. 

52557. 

52557. 

52557 

52459. 

52459. 

52459. 

52459. 

52459 

COL 


ROW  5 


;ii 

il2 
'22 


52554. 

52554. 

52554. 

52554. 

52554 

52557. 

52557. 

52557. 

52557. 

5  2557 

52459. 

52459. 

52459. 

52459. 

52459 

NOTE:      Units  are  micro  inches-per-inch. 
52,500. 


Correct  values  are  E 


ij 


Table  14.  Strain  from  Taylor  series, 
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COL 


ROW  1 


COL 


ROW  2 


COL 


ROW  3 


COL 


ROW  M 


COL 


ROW  5 


:i2 

^22 


^22 


=  12 
^22 


,11 

i;i2 

=22 


^1 1 

:i2 

^22 


1 


51241, 
47634. 
51151. 

1 


51241 , 
47634, 
51151  . 

1 


51241, 
47634, 
51151. 

1 


51241  . 
47634. 
51151  . 

1 


51241, 
47634, 
51151. 


51241, 
47634. 
51151. 


51241 , 
47634, 
51151  , 


51241, 
47634, 
51151. 


51241 , 
47634. 
51151  . 


51241, 
47634, 
51151. 


51241. 
47634, 
51151. 


51241  , 
47634, 
51151  . 


51241  . 
47634. 
51151. 


51241 , 
47634, 
51151  . 


51241 , 
47634. 
51  151  . 


51241. 
47634. 
51151. 


51241 , 
47634. 
51151  . 


51241. 
47634, 
51151  . 


51241 , 
47634. 
51151  . 


51241. 
47634. 
51151. 


^°^^-'   Units  are  micro  inches-per-inch.   Correct  values  are  E 
=  51,190,  E^2  =  ^2}    =  47.582. 


51241, 
47634, 
51151. 


51241 , 
47634, 
51151  . 


51241, 
47634, 
51151. 


51241 . 
47634, 
51.151  , 


51241, 
47634, 
51151. 


11 


^22 


Table  15.  Deformation  gradient. 
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COL 


ROW   1 


COL 


ROW  2 


COL 


ROW  3 


COL 


ROvV   4 


COL 


ROW  5 


:,ii  • 

J2' 
'22- 


■  11 

■21 
"22 


■  11 
■12 
■21 

'22 


J2 
^21 
^22 


''ll 
■22 


1 


-0.56849 

-0.93425 

0.88427 

-0.48175 

1 


-0 
-0 
0. 
-0, 


56849 
93425 
88427 
48175 

1 


-0.56849 

-0.93425 

0.88427 

-0.48175 

1 


-0.56849 

-0.93425 

0.88427 

-0.48175 

1 


-0.56849 

-0.93^25 

0.88427 

-0.48175 


NOTE:   Correct 


0.88433,    F 


22 


values  are 
-0.48170. 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93^25 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


.56849 
.93425 
,88427 
.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


Fn    =  -0.56830,   F^2  =  "0.93433,    F 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 


-0.56849 

-0.93425 

0.88427 

-0.48175 
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Table  16,   Green  deformation  tensor. 


7^1 


COL 


ROW  1 


COL 


ROW  2 


COL 


ROW  3 


COL 


ROW  H 


COL 


ROW  5 


'11 

:i2 

'22 


:i2 

'22 


'11 

:i2 

'22 


:11 
^12 

'22 


'11 

:i2 

'22 


1 


1  , 
0, 
1  . 


10511 
10511 
10492 

1 


1  .10511 
0.10511 
1 .10492 

1 


1, 
0, 
1. 


10511 
10511 
10492 

1 


1 .10511 
0.1051 1 
1 .10492 

1 


1  , 
0. 
1  . 


10511 
10511 
10492 


NOTE:      Correct   values 


10511 
10511 
10492 


1  .10511 
0.10511 
1  .10492 


10511 
10511 
10492 


1  .10511 
0.10511 
1 .10492 


1  . 
0. 

1  . 


10511 
10511 
10492 


10511 
10511 
10492 

3 


1 .10511 
0.10511 
1 .10492 


1. 
0. 

1. 


10511 
10511 
10492 


1 . 1 05 1 1 
0.10511 
1  .10492 


10511 
10511 
10492 


10511 
10511 
10492 


1  .10511 
0.1051  1 
1  .10492 


1. 

0, 

1. 


10511 
10511 
10492 


1 .1051 1 
0.1051 1 
1 .10492 


1.10511 
0.10511 
1 .10492 


are  C^ ^    =   C^^  =    1.10500,    C. ^  =  0.10500 


1  , 
0, 
1  . 


10511 
10511 
10492 


1  .1051  1 
0.10511 
1  .10492 


1.10511 
0.10511 
1.10492 


1  . 1 05 1 1 
0.10511 
1  .10492 


1  .10511 
0.10511 
1.10492 
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Table 

17.      Right   stretch 

tensor. 

COL 

1 

2 

3 

4 

5 

ROW   1 
COL 

^22 

:             1.05005 
:             0.05005 
:             1.04996 

1 

1  .05005 
0.05005 
1.04996 

2 

1.05005 
0.05005 
1.04996 

3 

1.05005 
0.05005 
1.04996 

1.05005 
0.05005 
1.04996 

5 

ROW  2 
COL 

"l1 
^22 

:             1  .05005 
:             0.05005 
:            1.04996 

1 

1  .05005 
0.05005 
1.04996 

2 

1 .05005 
0.05005 
1  .04996 

3 

1.05005 
0.05005 
1.04996 

1  .05005 
0.05005 
1.04996 

5 

ROW   3 
COL 

^12 
"22 

1.05005 
0.05005 
1.04996 

1 

1.05005 
0.05005 
1 .04996 

2 

1.05005 
0.05005 
1.04996 

3 

1.05005 
0.05005 
1.04996 

4 

1  .05005 
0.05005 
1.04996 

5 

ROW   4 
COL 

^22- 

1.05005 
0.05005 
1 .04996 

1 

1 .05005 
0.05005 
1.04996 

2 

1 .05005 
0.05005 
1 .04996 

3 

1.05005 
0.05005 
1  .04996 

4 

1  .05005 
0.05005 
1  .04996 

5 

ROW  5 

"ir 

^22- 

1.05005 
0.05005 
1.04996 

1 .05005 
0.05005 
1 .04996 

1 .05005 
0.05005 
1.04996 

1.05005 
0.05005 
1.04996 

1 .05005 
0.05005 
1  .04996 

NOTE: 

Correc 

t  values  are  U 

1 1    =  U22  =    1 

.05000,    U^2 

=   0.05000. 

Table   18.      Rotation  tensor, 
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COL 


ROW    1 


COL 


ROW   2 


COL 


ROW  3 


COL 


ROW  4 


COL 


ROW  5 


M  1 
^22 


11 
Jl2 

21 
^22 


Ml 

,2 

^21 

^22 


11 
M2 
^21 
^22 


Ml 

Jl2 

21 

22 


1 


-0, 
-0 
0. 
-0. 


50011 
86596 
86596 
50011 

1 


-0.50011 

-0.86596 

0.86596 

-0.50011 

1 


-0.50011 

-0.86596 

0.86595 

-0.50011 

1 


-0.50011 

-0.86596 

0.86596 

-0.50011 

1 


-0.50011 

-0.86596 

0.86596 

-0,50011 


-0, 
-0 
0. 
-0, 


50011 
86596 
86596 
50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


0.50011 
0.86596 
0.86596 
0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0, 

-0, 

0. 

-0. 


50011 
86596 
86596 
50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0, 

-0. 

0, 

-0. 


50011 
86596 
86596 
50011 

3 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0, 
-0 
0, 
-0. 


50011 
86596 
86596 
50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


NOTE:      Correct  values  are  R 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


-0.50011 

-0.86596 

0.86596 

-0.50011 


0.86603,    R 


22 


-0.50000, 


11    =   -0.50000,    R^2  =   -0.86603,    R21 


Table  19.  Lagrangian  strain  from  deformation  gradient. 
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COL 

1 

2 

3 

i| 

5 

ROW   1 
COL 

^22 

52554. 

:               52557. 

52459. 

1 

52554. 
52557. 
52459. 

2 

52554. 
52557. 
52459. 

3 

52554. 
52557. 
52459. 

4 

52554. 
52557. 
52459. 

5 

ROW  2 
COL 

:               52554. 
:                52557. 
:                52459. 

1 

52554. 
52557. 
52459. 

2 

52554. 
52557. 
52459. 

3 

52554. 
52557. 
52459. 

H 

5255  4. 
52557. 
52459. 

5 

ROW  3 

COL 

Ei2 
£■22 

52554. 
52557. 
52459. 

1 

52554. 
52557. 
52459. 

2 

52554. 
52557. 
52459. 

3 

52554. 
52557. 
52459. 

52554. 
52557. 
52459. 

5 

ROW  4 
COL 

^11  = 
E,2: 
t22- 

52554. 
52557. 
52459. 

1 

52554. 
52557. 
52459. 

2 

52554. 

52557. 
52459. 

3 

52554. 
52557. 
52459. 

4 

52554. 
52557. 
52459. 

5 

ROW  5 
NOTE :      I 

e": 
E22. 

Jn  1  ts 

52554. 
52557. 
52459. 

52554. 
52557. 
52459. 

52554. 
52557. 
52459. 

52554. 
52557. 
52459. 

52554. 
52557. 
52459. 

52,500, 


Correct  values  are  E 
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CHAPTER  IV 
EXPERIMENTS 


IV.  1   Test  Specimen 

The  prismatic  beam  shown  in  Figure  20  was  used  for  all  experi- 
ments. A  clear  plastic.  CR-39.  was  chosen  for  its  relatively  low 
modulus  of  elasticity  and  capacity  for  large,  elastic  strain. 

The  11X11  test  pattern  on  the  beam  was  generated  by  the  synthe- 
tic image  generator  and  applied  by  contact  printing.  First,  a  hard 
copy  of  the  pattern  was  photographed  using  Kodalith  film.  After 
coating  the  beam  with  a  buffer  of  polyurethane  and  naphtha,  a  Liquid 
Light  emulsion  was  applied,  contact  printed,  and  chemically  fixed 
(see  Figure  21  ). 

IV. 2  Test  Equipment 

Three  experimental  set-ups  were  configured;  two  were  for  strain 
tests  and  the  third  for  rigid-body  motion. 

The  strain  set-up  was  a  typical  photoelastic  load  frame  with 
special  fixtures  (see  Figure  22).  The  fixtures  were  for  four-point 
bending  (see  Figure  23)  and  cantilever  beam  (see  Figure  2i4)  experi- 
ments. 

The  equipment  shown  in  Figure  25  was  used  for  the  rigid-body 
motion  tests.  A  photocopy  of  a  protractor  was  taped  to  the  back  of 
the  lens  holder  and  rotation  was  measured  relative  to  the  tip  of  the 
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Figure  20.  Beam  fabricated  from  CR-39, 
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Figure  21.   Spot  pattern  on  be 


am. 
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Figure  22.  Experimental  set-up  of  load  frame. 
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Figure  23.  Set-up  for  four-point  bendi 


ng. 
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Figure  24.     Set-up   for  cantilever  beam. 


Figure  25.   Rigid-body  motion  set-up, 


82 


clamped  pointer.  Measurements  were  accurate  to  about  1  degree.  A 
dial  indicator  measured  translation  in  the  horizontal  direction  with 
an  accuracy  of  ±0.0005  inch. 

Since  image  processing  equipment  was  unavailable  at  the 
University  of  Florida,  the  equipment  for  the  rigid-body  experiments 
was  transported  to  USC  where  the  digital  images  were  recorded. 

IV. 3  Experimental  Procedure 

Unfortunately,  the  strain  tests  were  not  done  because  the  load 
frame  was  too  large  to  transport  to  USC.  The  second  order  model  in 
equation  (53)  was  for  these  tests. 

The  equipment  shown  in  Figure  25  and  an  image  processing  system 
similar  to  the  one  shown  in  Figure  3  (based  on  the  IBM  PC)  was  used 
for  the  rigid-body  experiments.  The  CCD  camera  was  a  Sony  model 
XCM-38  with  a  zoom  lens,  and  the  digitizer  board  was  an  eight-bit 
Datacube  "Frame  Grabber"  with  385  rows  and  374  columns  of  pixels. 

The  set-up  included  several  routine  checks.  First,  alignment 
between  the  rail,  table,  and  camera  were  checked  to  minimize  experi- 
mental error.  Then,  the  zoom  lens  was  adjusted  to  enlarge  the 
pattern  image  as  much  as  possible.  Resolution  in  the  horizontal 
direction  was  measured  at  358  pixels  per  inch. 

Lighting  was  adjusted  to  provide  the  usual  illumination  for 
tests  in  the  USC  lab,  and  the  transparent  specimen  caused  some 
lighting  problems.  Contrast  was  impaired  by  the  light  transmitted 
through  the  beam. 
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After  set-up,  images  were  digitized  and  stored  on  floppy  disk. 
All  images  were  transferred  to  magnetic  tape  via  USC's  VAX-1 1/780 
computer  for  transport  to  the  University  of  Florida. 


CHAPTER  V 
ANALYSIS  OF  RESULTS 


The  analysis  of  results  has  been  divided  into  two  sections: 
numerical  and  experimental.  The  numerical  section  summarizes  synthe- 
tic image  work  and  the  experimental  section  summarizes  the  analysis 
of  the  images  digitized  at  USC. 

V. 1  Numerical 

The  analysis  of  synthetic  images  included  rigid-body  motion, 
homogeneous  strain,  combined  motion,  and  the  influence  of  noise,  gray 
levels,  and  spot  radius.  To  ensure  a  good  random  sample,  the  results 
from  a  variety  of  spot  spacings  were  averaged.  The  pattern  was  5X5 
with  the  icon  being  spot  (3,3).  Except  as  noted,  the  gray  level 
difference  was  200,  and  the  zero-mean  noise  was  Gaussian  with  a 
standard  deviation  of  7.0  gray  levels.  The  absolute  values  of  noisy 
pixels  with  values  less  than  zero  were  used  instead  of  the  negative 
gray  levels. 

In  the  following  tables,  the  average  absolute  error  is  defined 
as  follows:  average  absolute  error  is  the  average  of  the  absolute 
values  of  the  errors.  For  example,  if  E^  •,  =£^2=^22"^^  ^'^'^  ^^^  computed 
values  are  E^^=100p,  Ei2=-'l50ij,  and  E22=50vi,  the  average  absolute 
error  is  lOOy. 
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Standard  deviation,  also  in  the  tables,  is  the  standard  devia- 
tion of  the  computed  values.  For  the  strain  example  above,  the 
standard  deviation  is  I32u. 

For  translations  of  various  amounts  in  the  x  and  y  directions, 
the  average  absolute  error  was  0.008  pixels  with  a  standard  deviation 
of  0.011  pixels.  Translations  near  whole  pixels  had  slightly  less 
error.   (Equation  (38)  suggested  a  spatial  sensitivity.) 

Rigid-body  rotation  was  about  the  image  center  which  was  approx- 
imately two-thirds  of  a  pixel  below  the  pattern  center.  Rotations 
ranged  from  0  to  ±180  degrees,  and  the  results,  computed  using  the 
elements  of  the  rotation  tensor,  were  very  consistent  with  an  average 
absolute  error  of  0.0039  degrees  and  a  standard  deviation  of  0.0008 
degrees. 

Several  strain  cases  were  investigated  and  summarized  in  Table 
20.  Normal  strains  were  E  =E   with  E  =E  =0;  shear  strains  were 
E  =E22=0  with  E  =E   ;  and  combined  strains  were  E  =E  =E  =E   . 
See  equation  (12)  for  strain  definitions. 

Noise  had  a  significant  effect  on  results.  As  suggested  by 
equation  (38),  increasing  SNR  decreased  translation  and  strain  error 
(see  Table  21).   Framing  failures  began  occurring  below  SNR=100. 

Gray  level  was  another  significant  variable  because  of  the  SNR 
and  the  digitization  of  illumination  intensity.  For  constant  noise, 
increasing  gray  level  difference  increased  the  SNR  and  thereby  in- 
creased accuracy.  With  a  noise  standard  deviation  of  7.00  gray 
levels,  spots  were  frequently  unidentifiable  below  a  gray  level 
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Table  20. 

Summary  of  strain 

analysis  for 

synthetic  images. 

Displacement 

Strain 

Strain 

Gradient 

Ave.  Abs. 

Standard 

Case 

au^/aXj 

Correct 

Error 

Deviation 

Normal 

0.000010 

10 

17 

10 

0.000100 

100 

17 

15 

0.001000 

1001 

22 

27 

0.010000 

10050 

40 

32 

0.100000 

105000 

40 

49 

Shear 

0.000010 

10 

24 

0 

0.000100 

100 

29 

0 

0.001000 

1000 

25 

0 

0.010000 

10000 

33 

0 

0.090000^ 

90000 

34 

0 

Combined 

0.000010 

10 

3 

14 

0.000100 

100 

7 

28 

0.001000 

1000 

23 

26 

Limited  by  pattern  geometry. 
NOTE:   Rigid-body  motion  was  zero. 
par-inch.  SNR=8l6. 


Strain  units  are  micro  inches- 
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Table  21.  The  effect  of  SNR  on  translation  and  strain  calculations, 


Trans 

lation 

Strain 

Ave.  Abs. 

Standard 

Ave.  Abs 

Standard 

SNR 

Error 

Deviation 

Error 

Dev 

i at  ion 

100  • 

0.0049 

0.0362 

39 

58 

200 

0.0029 

0.0253 

19 

42 

400 

0.0021 

0.0176 

20 

35 

600 

0.0016 

0.0143 

14 

32 

800 

0.0012 

0.0123 

10 

26 

1000 

0.0012 

0.0109 

8 

21 

2500 

0.0008 

0.0068 

5 

8 

5000 

0.0006 

0.0043 

3 

6 

10000 

0.0004 

0.0034 

2 

5 

00 

0.0001 

0.0001 

0 

0 

NOTE:   Translations  were  10.5  pixels  in  x 
strains  were  E..=100y. 


and  y.   No  rotation.   All 


Table  22.  The  effect  of  gray  level  difference  on  translation  and 
strain  calculations. 


Translation 

Strain 

Ave.  Abs. 

Standard 

Ave 

.  Abs 

Standard 

H 

Error 

Deviation 

Error 

Deviation 

32 

0.0257 

0.0270 

** 

»» 

64 

0.0043 

0.0349 

66 

62 

128 

0.0026 

0.0172 

20 

45 

256 

0.0009 

0.0089 

3 

16 

512 

0.0007 

0.0043 

5 

8 

1024 

0.0003 

0.0022 

4 

5 

2048 

0.0002 

0.0011 

2 

2 

4096 

0.0001 

0.0006 

1 

2 

NOTE:   Translations  were  10.5  pixels  in  x  and  y, 
strains  were  E.  •  =  i00p. 


No  rotation.   All 
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difference  of  32.  Below  a  difference  of  6H ,  a  strain  of  E..=100p  was 
undetectable  (see  Table  22). 

An  increase  in  gray  level  difference  reduced  the  error  caused  by 
digitization.  Without  noise,  a  difference  of  16  gray  levels  produced 
an  average  absolute  strain  error  of  12y  (for  E..=100y)  with  a  stand- 
ard deviation  O.OSu  and,  above  a  difference  of  6^,  error  was  nil. 
However,  SNR  effects  greatly  overshadowed  digitization  error. 

The  effect  of  spot  radius  decreased  rapidly  with  increased 
radius  (see  Table  23).  Equation  (38)  suggested  this  for  small  5, 
and,  in  fact,  almost  no  change  occurred  for  radii  above  10  pixels. 


Table  23.  The  effect  of  spot  radius  on  translation  and  strain 
calculations. 


Translation 

Strain 

Spot 

Ave.  Abs. 

Standard 

Ave.  Abs 

St 

andard 

Radius 

Error 

Deviation 

Error 

Deviation 

2 

0.0030 

0.0233 

29 

20 

1 

0.0018 

0.0272 

10 

32 

6 

0.0013 

0.0085 

7 

15 

8 

0.0016 

0.0069 

8 

13 

10 

0.0020 

0.0052 

8 

19 

12 

0.0021 

0.0059 

5 

11 

NOTE:   Translations  were  10.5  pixels  in  x  and  y.   No  rotation.   All 
strains  were  E^.=100y.   SNR=816. 


How  much  VAX  11/750  computer  time  did  the  analysis  of  a  256X256 
pixel  image  require?   For  a  3x3  spot  pattern,  CPU  time  was  about  ^1.9 
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seconds  with  no  printout  written  to  disk  and  about  6.5  seconds  with  a 
full  printout  (see  Tables  1-9  and  12-19).  An  11X11  pattern  required 
about  14.9  seconds  for  no  printout  and  about  28.3  seconds  for  a  full 
printout.  Due  to  overall  system  load,  CPU  times  varied  as  much  as  15 
percent. 

V.2  Experimental 
Images  digitized  at  USC  were  for  rigid-body  experiments.  These 
included  rotations  of  1.0.  lo.O.  and  63.5  degrees;  horizontal  (y 
direction)  translations  of  1.00.  10.00.  and  10.50  pixels;  and  a  com- 
bined motion  of  30.0  degrees  rotation  plus  10.00  pixels  translation 
in  the  horizontal  direction.  The  images  digitized  during  the  experi- 
ments  were  summarized  in  Table  24. 


Table  24.   images  used  for  rigid-body  motion  experiments, 


Gray  Level 

Image 

Black 

White 

% 

H 

SNR 

Undeformed 

U005 
U006 
U007 

57 
88 
50 

103 

151 

95 

3.72 
5.16 
2.49 

46 
63 
45 

153 
149 
327 

Deformed 

0005 
D006 
D007 
D008 
D009 
D010 
D011 

NOTE;  Grav  1 svp 

91 

96 
96 
82 
81 
86 
48 

1  .<?    rr     3>-iH  U 

150 
158 
158 
147 
145 
151 
90 

2.98 
3.56 
3.18 
4.57 
4.42 
4.26 
2.80 

59 
62 
62 
65 
64 
65 
42 

392 
303 
380 
202 
210 
233 
225 
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In  spite  of  relatively  low  SNR,  the  images  were  still  usable. 
The  SNR  for  the  images  were  near  the  minimum  usable  level  suggested 
by  synthetic  image  analysis.  In  fact,  undeformed  images  U005  and 
U006  required  a  simple,  ad  hoc  adjustment  to  one  of  their  mapping 
parameters.  Otherwise,  analysis  proceeded  as  expected. 

Rotations,  accomplished  using  the  lens  holder  previously  de- 
scribed, were  about  a  point  near  the  image  center.  As  shown  in  Table 
25,  the  average  results,  except  for  the  third  test,  were  within  the  1 
degree  accuracy  of  the  experiment. 

The  average  and  the  average  absolute  strain  error  was  computed 
using  all  the  strains  (i.e.,  E-,  •,  ,  E^2»  ^"<^  ^22  ^^  ^"'""^  ^^^  spots) 
computed  by  the  analysis  program.    The  actual  strain  for  any 


Table  25.  Results  of  rigid-body  rotation  experiments. 


Rotation 

Strain  Error 

Image 

Actual 

Ave. 

Error 

Std.  Dev. 

Ave. 

Ave.  Abs. 

D005 

1.00 

0.81 

-0.19 

0.03 

-119 

336 

D006 

10.00 

9.23 

-0.77 

0.02 

360 

529 

D007 

63.50 

64.83 

1.33 

0.03 

191 

3737 

D011 

30.00^ 

30.49 

0.49 

0.02 

1661 

2298 

30.00  degrees  plus  10.00  pixels  horizontal  translation. 
NOTE:  The  undeformed  image  was  U005  for  the  first  three  experiments 
and  U007  for  the  last.  All  rotations  were  in  degrees.   Strain  units 
are  micro  inches-per-inch.  Measurement  accuracy  was  about  1  degree. 
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rigid-body  motion  was,  of  course,  0.0.  Although  the  noise  was 
Gaussian,  the  average  strain  error  was  not  the  expected  value  of 
0.0.  The  reason  for  this  was  imaging  system  distortion. 

Average  results  obtained  during  translation  experiments  are 
summarized  in  Table  26.  The  first  test  was  within  the  ±0.18  pixel 
resolution  of  the  set-up  and  the  other  two  tests  were  not.  Evidently 
there  was  a  slight  "tilt"  between  the  axes  of  the  lens  holder  and  the 
camera. 

There  was  good,  linear  consistency  between  the  three  translation 
tests.  The  quotient  of  the  second  translation  divided  by  10.0  was 
1.0^1  and  the  third  by  10.5  was  1.047.  This  consistency  suggested 
some  systematic  error  in  the  experimental  measurement  probably  caused 
the  increased  error  in  the  latter  tests. 

An  apparent  "distortion"  existed  in  all  images,  and  the  effect 
of  this  was  manifested  as  high  strain  error.  This  was  especially 
noticeable  in  the  average  strain  error  which  had  an  expected  value  of 
0.0.  Since  the  tests  were  not  preceeded  by  extensive  calibration, 
the  exact  source  cannot  be  positively  identified.  No  doubt  the  dis- 
tortion was  induced  by  the  image  processing  system. 

Other  sources  of  error  were  low  SNR,  low  gray  level  difference, 
and  relatively  coarse  resolution  of  the  experiment  set-up.  Although 
a  common  solution  for  minimizing  noise  effects  is  averaging  multiple 
images,  only  one  copy  of  each  image  was  used  in  these  experiments.  A 
small  increase  in  gray  level  difference  was  possible  through  the  use 
of   additional   lighting   (or   by   using   an   opaque,   reflective 
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specimen).  Improved  resolution  was  not  possible  due  to  the  limita- 
tions of  available  equipment. 

The  aaalysis  program  made  efficient  use  of  computer  resources. 
Typical  CPU  time  for  a  real  image  with  no  printout  written  to  disk 
was  about  25  seconds  and  about  38  CPU  seconds  for  full  printout. 

Although  not  implemented  in  this  study,  it  was  possible  to 
further  decrease  CPU  time.  For  example,  a  simple  strain  definition 
such  as  change  in  gage  length  divided  by  gage  length  and  estimating 
rigid-body  rotation  by  icon  rotation  would  have  saved  about  12  CPU 
seconds.  Additional  savings  of  about  6  CPU  seconds  per  deformed 
image  (for  images  with  the  same  undeformed  image)  were  possible  if 
the  undeformed  image  were  analyzed  only  once. 


CHAPTER  VI 
CONCLUSIONS  AND  RECOMMENDATIONS 


VI .  1   Conclusions 


The  numerical  and  experimental  results  successfully  demonstrated 
the  potential  of  pattern  mapping  in  motion  analysis.  With  this 
method,  it  was  possible  to  obtain  accurate  measurements  of  rotation, 
translation,  and  strain  for  a  variety  of  plane  motions  (see  Tables 
20-26).  Automation  minimized  user  time  and  required  only  a  modest 
amount  of  computer  resources.  Whole  image  analysis  was  also 
thoroughly  demonstrated. 

Segmentation  algorithms  performed  well  in  all  images  tested. 
The  estimation  of  histogram  parameters  was  highly  efficient  and  pro- 
vided adequate  results  without  user  interaction.  Except  for  very  low 
SNR,  automatic  framing  was  easily  accomplished. 

Mapping  algorithms,  designed  to  produce  unique  mapping,  worked 
well  for  a  large  variety  of  motions.  Although  low  SNR  prevented 
automatic  mapping  in  some  images,  no  erroneous  maps  were  produced. 
In  two  of  the  experimental  images,  low  SNR  necessitated  a  manual 
reset  of  one  of  the  parameters.  Averaging  multiple  images  or 
filtering  may  have  helped  the  SNR  problem. 

The  most  important  variables  were  SNR  and  gray  level  difference 
with   higher   values   yielding   better   accuracy.     Numerical   and 
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experimental  results  suggested  a  practical  ninimura  SNR  between  100 
and  200.  Although  a  lack  of  data  prevents  a  direct  comparison  of 
numerical  and  experimental  results,  they  agreed  "directionally"  for 
rotation  and  translation. 

In  general,  spot  radius  was  not  an  important  variable.  Practi- 
cal experience  and  numerical  results  indicated  radii  should  be  at 
least  4  to  6  pixels. 

Pattern  mapping  has  the  potential  for  significant  data  compres- 
sion. Other  image  processing  applications  require  archiving  entire 
images.  If  the  pattern  is  well  suited  to  the  application,  the  entire 
displacement  field  may  be  adequately  reconstructed  from  the  spot 
information. 

VI.  2  Recommendations 

Although  the  experimental  data  included  in  this  study  is  encour- 
aging, pattern  mapping  needs  extensive  experimental  verification  with 
a  wide  variety  of  plane  motion  applications.  Only  by  using  a  rigor- 
ously calibrated  system  and  analyzing  many  images,  can  the  full 
potential  of  this  method  be  known. 

A  logical  next  step  is  the  consideration  of  image  processing  and 
pattern  recognition  tools  such  as  filtering,  local  averaging,  other 
dimensions  of  feature  space,  etc.  These  may  significantly  improve 
performance  in  low  SNR  situations  or  lead  to  the  discovery  of  new 
parameters  and  more  general  mapping  theories.  The  implementation  of 
the  full  syntactic  recognition  theory  should  also  be  investigated. 
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An  extension  of  this  method  to  three-dimensional  motion  and 
mapping  of  multiple  images  (mosaics)  is  also  suggested.  Three- 
dimensional  images  that  are  not  restricted  by  camera  view  have  much 
potential  for  practical  application. 

As  a  final  note,  it  is  suggested  that  pattern  mapping  be  inte- 
grated with  other  experimental  methods  and  numerical  analysis 
tools.  This  includes  advanced  graphics  for  data  manipulation  and 
presentation. 
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